I had a problem show up on some of our servers. Visiting sites would work fine, but as soon as you log in to a WordPress site, then your IP was blocked at the firewall level. It took a bit of hunting around the OSSEC logs to figure out the cause, and then finally tipped off to a local rule to combat the blockage. I outline the process of figuring out what was wrong, and how to fix it below.
So initially, this was quite confusing. All of a sudden people would have their IP blocked. I checked the different sites, and seemed to have no problem. Then when I logged in to the back end, BAM, blocked as well. We have Shorewall running, so doing:
Showed all of the IPs that Shorewall had blocked. This could also be done by using iptables
Sure enough, my IP had been blocked. I unblocked my IP with:
Or could also do
where INPUT is the position or section of the firewall chain, and “2” is the line number containing my IP address.
Then I checked with other web applications on that server. Where they also causing an issue? I logged in to an Omeka install. No problems.
OK. I know OSSEC is to blame some how. It’s an awesome HIDS (Host Intrusion Detection Software) that actively responds to issues on the server based on scanning through the system logs and various rules.
OSSEC keeps itself chroot’ed to /var/ossec/, so all of the ossec logs are located in /var/ossec/logs/.
I first looked in the /var/ossec/logs/active-responses.log. Sure enough, a couple of lines like this show my IP being completely blocked to the server.
So, there we are. OSSEC blocking the IP for some reason. Now why is it blocking the IP?
Taking a look in the /var/ossec/logs/alerts/alerts.log file to see why it thinks it needs to block the IP…
There were other lines in there with my IP, but nothing would/should have caused blocking, like a WordPress login event, or an SSH login event. Also, the error above is categorized as an IDS event with level 6, which by default OSSEC rules means it gets blocked.
HOW TO FIX IT!
As a quick fix, I changed the “suhosin.memory_limit” option in /etc/php.d/suhosin.ini to 256M, and the “memory_limit” in /etc/php.ini to 256M so that no error would be generated.
Now came the hard part of finding out how to fix it for real. OSSEC is a pretty big beast to tackle, so I turned to my friendly web search engine to help out.
To fix the issue, I would need to write a decoder or new rule to ignore the suhosin rule causing the problem. OSSEC has descent documentation to get you started, but fortunately the blog linked above had the solution already. https://www.atomicorp.com/forum/viewtopic.php?f=3&t=5612
From user ‘faris’ in the forum linked above:
Add the following lines the the /var/ossec/etc/rules.d/local_rules.xml file.
What these new rules do is change the level of the default rules that were tagged/decoded as being suhosin errors. In the first rule, if the default error is 20100, and is decoded (or tagged, or matches) as suhosin, then set the level to 14 instead of the default 8.
The second rule detects if the default error 20101 is decoded as coming from suhosin and sets the level to 12 instead of the default 6.
The third new rule looks at any error tagged as suhosin and if the error has the matching text in it, then it sets the error level to 5 (below the limit for firing an active response).
So, just add that group of rules to the local_rules.xml file and restart the OSSEC service. BA-DA-BING! no more blocking the IP when logging in to WordPress.