Hi everyone,as i m new at website ..i have it maybe a half year..and even i got unlimited bandwidth but that doesnt really exist..i m worried about google bots that drives my bandwidth.It start to happen few day ago where in few days bots from Coldwater and mountain view already use 13 gb of my bandwidth..what is happening..anyone ? do add there is bingboot and facebookexternal but bots from google coldwater are always on my site..20 of them currently plus few from mountain view What should i do ? why is this happening ? My site is here footballtarget co m Regards
You can create a file called robots.txt or modify your existing one to let Google and other bots know to pause between requesting documents on your server. You would need to add something like this: User-agent: * Crawl-delay: 10 You can also control Google's crawl rate using Google's Webmaster Tools site: http://www.google.com/webmasters/tools/
Make an account with google webmaster tools and lower the crawl rate for your site. That should help reduce the bandwidth used by google bots.
Hi ...i have of course account with webmaster...google has set there own crawl rate ( called recommended).But if lower this rate trough webmaster would that have some negative impact on my site? Do you have some idea,why this happen in the first ? As google crawl my site million time..i cant see the reason why everyday some 100 bots coming up ? Thx