Solution: Bots with no user agent eating up bandwidth

Last night, i faced a server issue, that a robot with empty user agent was eating my CPU (i guess) and bandwidth.

Similar issue was discussed on this thread

So, i found a solution to block such request on apache level.


Blocking empty user agents is simple in lighttpd. Edit your lighttpd.conf as follows:

Ensure that mod_access is loaded:

server.modules = (    
 … other modules …

Add the following line:

$HTTP["useragent"] == "" 
{ url.access-deny = ( "" )}

Reload the lighttpd configuration and you’re done.


Enable mod_rewrite and add the following to your configuration:

RewriteEngine on 
RewriteCond %{HTTP_USER_AGENT} ^$ 
RewriteRule ^.* - [F]

more details here …


Share your thoughts & feedback

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: