Hi, I am using a special "images" folder with only images in it for a site. I "pull" images from that folder to all pages on my site. This is only for good-organization of my work. But Google results finds/shows this folder without "real content". And I don't want the folder to show up in Google. (it shows up when people type in site:www.myside.com) Probably I can ban that with robots.txt or with what? I only want to exclude the folder. I've been excluding certain pages like iframes, but never had to exclude an entire folder... Suggestions/help are warmly appreciated
There are bots specifically for images, so you could also disallow them entirely from your site (in case at some point you end up with images outside of /images/ directory). User-agent: Googlebot-Image Disallow: / User-agent: MSNBot-Media Disallow: / User-agent: YandexImages Disallow: / User-agent: YandexMedia Disallow: / User-agent: Baiduspider-image Disallow: / Code (markup):
OK, so I will disalow 'that foler', but that folder contains a bunch of images. How will this affect those images? Will anyone be able to find them via Google Images search? (because a large part of my traffic actually comes from image search) Thank you all for helping!
You should add description with HTML <img src="..." alt="..." /> It also be middle by alt tags while attaching image.
You don't need to disallow anything then if you want the images to show up in image search. Just don't make a link to the images folder or put an index file in there and you'll be fine. The images will still be indexed, but not the folder name as a "web page". You're over-thinking things.
You can use wildcards too...this is very useful. I recently had to prevent Google crawling aspx pages in tha folder but allow the html pages to be crawled, very useful. Also I don't think anyone has mentioned that Google webmaster tools is great for testing what disallows you have in the robots.txt. I usually use it to make sure that the pages I DO want crawling are still available to the Googlebot.
Actually I had same problem. Ya you can disallow these link using Robots.txt but user can still access your folder. So I would suggest you use .htaccess file. In that file use which will block all folders to access from smart user. I hope it will helpful to you. Thanks