617 MB traffic by bots ?

My traffic viewed by bots is more than the real traffic



what it is actually i could not understand

 

Your website's most of the bandwidth is occupied by Robots..then why don't you block these unwanted robots by robots.txt..

 

thanks for the reply +rep added

dont know abt robots.txt much. well i will block those bots now after getting some info on robots.txt

 

You can find the basic understanding of robots.txt at
Robots and spammy robots list

 

Very good info thanks a lot

 

This will happen pretty much no matter what you do.

The spammy bots will ignore the robots.txt and the ones you want to come around should suck down everything you have to offer.

I have a large amount of PDF's on my site and I allow Google to index every one because I get a lot of nice traffic from it.

When all is said and done each month all SE's pull about 1.5GB.

I own the server and line so it doesn't bother me the amount of BW used.

 

what kind of site are you running?

 

I know some spammy bots ignore robots.txt..in that case you can track their IP addresses from your traffic log and then block those bots using .htaccess...

I have applied it on my website and it gives perfect results...