What is google spider doing?

I have noticed my site is getting a lot of google and other spiders...but they seem to
only target useless tags in articles...why are spiders going after tags such as /go/
rather then spidering actual acticles that could be of benefit?

any ideas?

 

You're websites getting ready to drop in rankings from what I've been seeing. Some people say Google is reindexing websites and it's been causing havoc on many authority websites.

 

my sites are going to drop because tags are being spidered?

 

Get all the useful webpages indexed.
For your site, I advise you to stop the spider targeting the tags because it is useless completely.

 

You might have to use the robots.txt to indicate which parts google will index and which not.

 

Uh... What tags used for here?

 

Googlebots / Yahoo Slurp / MSN/Bing Bot = Common

Others = Check carefully first. Most of them are useless bots and some are content scraper bots.

You should block those unwanted bots from accessing your website. It would consume bandwidth and even copy your own contents. This is a good tutorial to do this - How to Block Unwanted Bots from Your Website with .htaccess

 

Thanks for that link. I really hadn't considered the idea of that, but I think you are correct.

 

Google bots are going nuts and they are not following robots.txt files on a lot of website. I assume you have a website using WP CMS and the /tag are getting indexed - you're not the only one.