Bandwidth Suckers - how to block?

Hi J.D.,

and thanks very much for your help.

I used Awstats from my CPanel and the attacks spread over various IPs. I tried blocking them all with CPanel's IP deny manager but a new one would then come up and in the mean time my BW limit was reached.

It's a site with very little traffic, a few photos and very little text, therefore not heavy at all. I don't understand the reason for picking this website of mine and really don't know what to do.

Do you have any ideas at all as right now my only option is to leave it down.

Thank you again J.D. and thanks to anybody else who might be able to help.

 

Pick a couple of IP addresses and find them in the logs. Things to look for:

* what user agent (browser) is reported - is it an automated tool or one of the standard browsers (can be easily forged, but will do for now)

* were they accessing same image over and over again or it looks like a normal access pattern (e.g. they grab the HTML and then everything HTML refers to, such as CSS, graphics, etc)

* what's the time between page accesses - is it something a human-like pattern (a few seconds between hits) or an automated tool-like?

What kind of bandwidth we are talking about - what's your current limit?

J.D.

 

OK, I'm looking into it.

My current BW limit is set at 675MB but the used BW is at 735 and the site is down.

I'll let you know.


I just noticed (from Awstats): 1 hit, 1 page 71.56MB!?!
How is that done?

 

700 MB/month is roughly 24 MB/day. If your average page is about 20 KB, then you will use up your limit after about 1000 page views (including all the graphics, CSS, etc).

If you are hitting this limit, you can configure your web server to handle content expiry better. For example, you can configure your server so that all images, CSS, JS, etc files can be cached by the client for about a day or so:

http://httpd.apache.org/docs-2.0/mod/mod_expires.html

That should be easy to track down - just search the logs.

J.D.

 

Who are you hosting from with all the bandwidth problems?
I check logs and add the offending ip's to the deny list in htaccess.
Also block some bots and images.
Foreign countries are big bandwidth suckers.

Maybe add these to your htacess file:

# Enable directory listing from this point
Options +Indexes

# Omit certain file types from the directory listing
IndexIgnore *.bmp *.gif *.jpg *.png *.jpeg *.txt

<Limit GET POST>
order allow,deny
allow from all
deny from .id
deny from .interpacket.net
deny from .lt
deny from .mk
deny from .my
deny from .ro
deny from .yu
deny from .sg
deny from .kr
deny from .za
deny from .in
deny from .si
deny from .br
</Limit>
<Limit PUT DELETE>
order deny,allow
deny from all
</Limit>

 

Apache will do double DNS lookup on every hit in order to handle these directives. This will imact your site's performance quite a bit.

J.D.

 

OK thanks.
Well what is the best way to ban these countires and ip blocks?

 

I usually block by offending IP address ranges, but may be there's some kind of a geo module available for this. Ask around.

 

Repeat offenders are by net blocks.
I am trying by Countries now.
I started a new thread for this.
And found some htaccess validators and posted them too in a new thread.

 

Try this for the bandwidth suckers!

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?yourdomain/.*$ [NC]
RewriteRule \.(gif|jpg|png|jpeg|txt|)$ - [F]

This will lock your images!!

 

oopps take out the | after txt

 

I am embarassed for this post, but it's right to let you know that my website was NOT under attack by BW suckers!

There was something wrong with this spider sniffer that caused errors while loading the pages and this would go on and on forever in an endless download of the page which in turn means that if someone let it go and not stop it it would suck MB after MB.

I realised that only when I looked at my own webpage and focused less on the stats. Actually I didn't even realize that I was denying IPs including mine, although that's because my IP changes everytime I connect.

So, in brief, I thank very much all those that helped and I'll keep in mind what was said for future reference (hoping I don't need it!).

I hope this post acts as a helpful warning for any non-expert just like me!

Thanks!