memory leak on nginx reload

[amorozkin]

RAM usage constantly grows on nginx -s reload

Having modsecurity rules loaded (even with modsecurity off) causes RAM usage to grow with each nginx -s reload and ultimately leads nginx to stuck with messages like:

Logs and dumps (/var/log/nginx/error.log)

Output of:
2020/08/06 20:00:20 [alert] 1962#1962: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2020/08/06 20:00:20 [alert] 1962#1962: sendmsg() failed (9: Bad file descriptor)
2020/08/06 20:00:20 [alert] 1962#1962: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2020/08/06 20:00:20 [alert] 1962#1962: sendmsg() failed (9: Bad file descriptor)
2020/08/06 20:00:20 [alert] 1962#1962: fork() failed while spawning "cache manager process" (12: Cannot allocate memory)
2020/08/06 20:00:20 [alert] 1962#1962: sendmsg() failed (9: Bad file descriptor)

To Reproduce

1. Configure nginx to load rules:
   /etc/nginx/nginx.conf

http {
...
   modsecurity off;
   modsecurity_rules_file /etc/nginx/modsec/main.conf;
..
}

2. Restart Nginx and check rules were loaded (/var/log/nginx/error.log):

2020/08/06 08:57:13 [notice] 13627#13627: ModSecurity-nginx v1.0.1 (rules loaded inline/local/remote: 0/911/0)

3. Generate load:

./nikto.pl -h https://your-site-name

4. Run several 'nginx -s reload' (with 3-4 minutes interval) and check RAM consumption with free -m command before and after nginx reload:

# free -m
              total        used        free      shared  buff/cache   available
Mem:           3951         433        2122          30        1395        3136
Swap:          2043          49        1994

# nginx -s reload

# free -m
              total        used        free      shared  buff/cache   available
Mem:           3951         451        2103          30        1395        3117
Swap:          2043          49        1994

# nginx -s reload

# free -m
              total        used        free      shared  buff/cache   available
Mem:           3951         464        2083          30        1404        3104
Swap:          2043          49        1994

# nginx -s reload

# free -m
              total        used        free      shared  buff/cache   available
Mem:           3951         481        2051          30        1417        3086
Swap:          2043          49        1994

.....

# free -m
              total        used        free      shared  buff/cache   available
Mem:           3951         901        1534          30        1515        2666
Swap:  

Expected behavior

'RAM used' should not steadily grow and should stay around the same level as it does for example without modsecurity rules loaded (in which case 'ram used' stays about 300 MB)

Server

• ModSecurity v3 master - 51d06d7 with nginx-connector v1.0.1
• WebServer: nginx-1.18.0
• OS : Ubuntu 16.04

Rule Set:

• https://github.com/coreruleset/coreruleset/archive/v3.3.0.tar.gz

Additional context

The same happens with modsecurity on in server's context.
Using SecResponseBodyAccess Off in modsecurity.conf

[albgen]

Same issue here. I have disabled modsec but i doubt this will be fixed...

[zimmerle]

Hi,

We are working on a lot regarding rules reload at: v3.1-experimetal. Is this issues happens on v3.0.4 or only in v3/master?

[amorozkin]

Hi, zimmerle, you are right - can't reproduce this with v3.0.4.

[hazardousmonk]

I'm still not able to resolve this. As nginx works with the ModSecurity + CRS ruleset, the memory size grows and grows until Nginx crashes. I've hacked around it by forcing nginx to restart as soon as it crashes, but that's a really bad fix. I tried to use the experimental version, however that didn't work out (the vhost on which ModSecurity is running was not working with the following error)[alert] 7955#0: worker process 9835 exited on signal 11 (core dumped) ...but thats for another topic. Any idea how I can get nginx and ModSecurity to behave?

nginx version: nginx/1.18.0
built by gcc 6.3.0 20170516 (Debian 6.3.0-18+deb9u1)
ModSecurity-nginx v1.0.1
ModSecurity v3/master

[albgen]

I'm still not able to resolve this. As nginx works with the ModSecurity + CRS ruleset, the memory size grows and grows until Nginx crashes. I've hacked around it by forcing nginx to restart as soon as it crashes, but that's a really bad fix. I tried to use the experimental version, however that didn't work out (the vhost on which ModSecurity was not working with the following error)[alert] 7955#0: worker process 9835 exited on signal 11 (core dumped) ...but thats for another topic. Any idea how I can get nginx and ModSecurity to behave?

nginx version: nginx/1.18.0
built by gcc 6.3.0 20170516 (Debian 6.3.0-18+deb9u1)
ModSecurity-nginx v1.0.1
ModSecurity v3/master

hi,
in my case, the issue, i think it was because i was loading the rules on each server directive which means many times as the number of server directives. Moving the loding of the rules on the main nginx.conf file fixed 2 issues: 1)The memory leak one which is the current issue and also 2)the slow start of nginx

[hazardousmonk]

I'm still not able to resolve this. As nginx works with the ModSecurity + CRS ruleset, the memory size grows and grows until Nginx crashes. I've hacked around it by forcing nginx to restart as soon as it crashes, but that's a really bad fix. I tried to use the experimental version, however that didn't work out (the vhost on which ModSecurity was not working with the following error)[alert] 7955#0: worker process 9835 exited on signal 11 (core dumped) ...but thats for another topic. Any idea how I can get nginx and ModSecurity to behave?
nginx version: nginx/1.18.0
built by gcc 6.3.0 20170516 (Debian 6.3.0-18+deb9u1)
ModSecurity-nginx v1.0.1
ModSecurity v3/master

hi,
in my case, the issue, i think it was because i was loading the rules on each server directive which means many times as the number of server directives. Moving the loding of the rules on the main nginx.conf file fixed 2 issues: 1)The memory leak one which is the current issue and also 2)the slow start of nginx

Unfortunately each vhost is running a different application, and the ruleset is fine tuned for each one. So loading the rules in the nginx.conf isn't something that's possible. Is the 3.0.4 or v3/master the latest stable version?

[albgen]

oh i see. That's a big issue.
If i'm not wrong, i read some times ago that this issue with the loading of the rules is being fixed but it will take a lot of time...
btw i always use v3/master

[zimmerle]

The last

I'm still not able to resolve this. As nginx works with the ModSecurity + CRS ruleset, the memory size grows and grows until Nginx crashes. I've hacked around it by forcing nginx to restart as soon as it crashes, but that's a really bad fix. I tried to use the experimental version, however that didn't work out (the vhost on which ModSecurity was not working with the following error)[alert] 7955#0: worker process 9835 exited on signal 11 (core dumped) ...but thats for another topic. Any idea how I can get nginx and ModSecurity to behave?
nginx version: nginx/1.18.0
built by gcc 6.3.0 20170516 (Debian 6.3.0-18+deb9u1)
ModSecurity-nginx v1.0.1
ModSecurity v3/master

hi,
in my case, the issue, i think it was because i was loading the rules on each server directive which means many times as the number of server directives. Moving the loding of the rules on the main nginx.conf file fixed 2 issues: 1)The memory leak one which is the current issue and also 2)the slow start of nginx

Unfortunately each vhost is running a different application, and the ruleset is fine tuned for each one. So loading the rules in the nginx.conf isn't something that's possible. Is the 3.0.4 or v3/master the latest stable version?

The last released version is 3.0.4.

The experimental branch is the 3.1-experimental

[zimmerle]

oh i see. That's a big issue.
If i'm not wrong, i read some times ago that this issue with the loading of the rules is being fixed but it will take a lot of time...
btw i always use v3/master

did you had the chance to test this particular issue in 3.1-experimental?

[albgen]

oh i see. That's a big issue.
If i'm not wrong, i read some times ago that this issue with the loading of the rules is being fixed but it will take a lot of time...
btw i always use v3/master

did you had the chance to test this particular issue in 3.1-experimental?

Nope. I can test it but it will take same take because it is not very deterministic the appeareance of the memory leak bug.

The slow start on the other hand can be tested. Will try tomorrow and let you know.

[zimmerle]

oh i see. That's a big issue.
If i'm not wrong, i read some times ago that this issue with the loading of the rules is being fixed but it will take a lot of time...
btw i always use v3/master

did you had the chance to test this particular issue in 3.1-experimental?

Nope. I can test it but it will take same take because it is not very deterministic the appeareance of the memory leak bug.

The slow start on the other hand can be tested. Will try tomorrow and let you know.

thank you.

[albgen]

hi zimmerle,

i cannot compile that branch...

cd /root
rm -rdf /root/ModSecurity/
sudo git clone --depth 1 -b v3/dev/3.1-experimental --single-branch https://github.com/SpiderLabs/ModSecurity
cd ModSecurity
sudo git submodule init
sudo git submodule update
sudo ./build.sh
sudo ./configure --with-maxmind=no
sudo make #here the errors

[zimmerle]

Can you confirm the last commit hash in your branch?