Hi, I'm getting a lot of unwanted traffic from URLs and with wicked keywords (irrelevant!), but from unrelated sites - e.g. warez/spam, adult sites and a ton of unrelated foreign sites. Has anyone seen anything similar? I'm getting this even to brand new sites that haven't been SEO-ed, even to domains that have just been connected to a hoster. Junk traffic without having done any SEO at all! I'm banning IP's and disavowing, but the way this traffic gets me is still a mystery... Any ideas on protecting against it?
you can read about this robots.txt and how to configure it correctly and not allow those spam. I'm here if you need any help