It's been two months now, I do not know why my robots.txt on webmaster showing Crawl-delay: 1 error. I just want to know how can I fix this issue on my WBT property?
This is really only required on huge websites but last I checked google doesn't pay any attention to that directive in robots.txt. I could be wrong but for google you set that literally in WMT. http://www.inmotionhosting.com/supp...tting-a-crawl-delay-in-google-webmaster-tools hope that helps, Nigel