Solution: Bots with no user agent eating up bandwidth

Last night, i faced a server issue, that a robot with empty user agent was eating my CPU (i guess) and bandwidth.

Similar issue was discussed on this thread
http://www.namepros.com/web-hosting-discussion/588889-unknown-bots-eating-up-my-valuable.html

So, i found a solution to block such request on apache level.

lighttpd

Blocking empty user agents is simple in lighttpd. Edit your lighttpd.conf as follows:

Ensure that mod_access is loaded:

server.modules = (    
"mod_accesslog",    
"mod_access",
 … other modules …
)

Add the following line:

$HTTP["useragent"] == "" 
{ url.access-deny = ( "" )}

Reload the lighttpd configuration and you’re done.

Apache

Enable mod_rewrite and add the following to your configuration:

RewriteEngine on 
RewriteCond %{HTTP_USER_AGENT} ^$ 
RewriteRule ^.* - [F]

more details here …
http://johannburkard.de/blog/www/spam/block-empty-user-agent.html

Share your thoughts & feedback

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: