Referrer spam
by CyberShadow on Feb.04, 2011, under Website
I got tired of pesky referrer spambots junking up my logs, so I decided to find a way to get rid of them. Turns out, almost all referrer spam on my blog is generated with a small number of User-Agents:
Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; Media Center PC 6.0; InfoPath.2; MS-RTC LM 8)
Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 3.5.30729)
Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.221.7 Safari/532.2
Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; en) Opera 8.50
Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3
Opera/9.64(Windows NT 5.1; U; en) Presto/2.1.1
Mozilla/5.0 (X11; U; Linux i686; it-IT; rv:1.9.0.2) Gecko/2008092313 Ubuntu/9.25 (jaunty) Firefox/3.8
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.14) Gecko/2009082707 Firefox/3.0.14 (.NET CLR 3.5.30729)
All of these refer to outdated browsers, very rare configurations (the ones with non-English language codes), and genuinely fake UA strings.
March 30th, 2011 on 1:49 pm
And what you’re doing with those user-agents? anyway to block it?
I’ve seen some posts you’ve made in serverfault about apache bringing down your entire server, and a solution involving oom. Is that still working? Same is happening here when i get DoS’ed.
Cheers,
March 30th, 2011 on 1:51 pm
Well, I just filter them from my logs and statistics, that’s all I care about. You can block them from accessing your website using .htaccess, but they’ll still show up in your logs (and log-generated statistics).
No
I replaced it with a cronjob to kill Apache if it’s not responding.
March 30th, 2011 on 2:10 pm
I see… It’s amazing how this can still happen, i’ve been fighting apache for 2 weeks now…
Have you ever tried mod_bw maximum connections limit? I’m experimenting with it now, it might be a good solution, at least for the simplest DoS attacks…
Would you care to share the cronjob you’re using to kill apache when unresponsive?
Cheers!
March 30th, 2011 on 2:19 pm
I haven’t tried any bandwidth-limiting modules. This is my cronjob:
wget -qT 15 http://localhost/index.html -O /dev/null || ( echo Uh-oh ; killall -9 apache2 ; sleep 5 ; /etc/init.d/apache2 zap ; /etc/init.d/apache2 start )March 30th, 2011 on 2:25 pm
mod_bw seems to be able to block a maximum number of conections per ip, and still not block the actual bandwidth, so it might work…
mod_evasive also should work, but for some reason it is not blocking any dos’er from my server…
anyway, thanks for the cronline and good luck