Stopping google from indexing

I noticed google has taken 20 or so of my pages out of their index, this might be cause of youtube feature I put in place so they are thinking it is duplicate content.


Is there anyway I can do something to stop the spiders from indexing that part of my site. The extension to the root domain is forum/vBTube.php and I dont want anthing from there or anything beyond that, ie forum/vBTube.php/video12365 indexed by the spider.


Can anyone help?

 

1 file - robots.txt

 

in the google webmaster tools there is the possibility to exclude some of your pages from google indexing even without using robots.txt

 

Thanks schizzorl86, I found the bit in the webmasters tools, I also discovered my images folder wasn't being indexed so no google image traffic

thanks again.

 

youtube images are FLV / flash items ... and as far my knowledge goes , it is something SE's are not at ease with .... so i doubt it is because of duplicate content as such .

 

Nice tips... Actually I did not try that.....but maybe later on when I encountered a problem like that.

 

You can do it using the robot exclusion protocol, but you probably don't want to. Google won't mark an entire page as duplicate for having a video, if the rest of the page is unique. That suggests other reasons for the pages dropping out of their index.

 

Read this how you can tell Google bots not to read certain files/folders of your website http://www.cosmocentral.com/page/robotstxt-Setup-for-Search-Engine-Spiders.aspx

 

use robots.txt to block spiders accessing certain parts of your website

 

Hi Mmecca, you can do that by editing your robots.txt file. Here's a Robots.txt Guide from our blog. I hope you'll find it informative...

 

use robots.txt file for not indexing. Sitemaps for indexing and submit it to google webmaster tools.