How Do I Ban Google From Seeing A Folder?

Discussion in 'robots.txt' started by Mr.Dog, Jan 16, 2013.

  1. #1
    Hi,

    I am using a special "images" folder with only images in it for a site. I "pull" images from that folder to all pages on my site. This is only for good-organization of my work.

    But Google results finds/shows this folder without "real content". And I don't want the folder to show up in Google.
    (it shows up when people type in site:www.myside.com)

    Probably I can ban that with robots.txt or with what?

    I only want to exclude the folder. I've been excluding certain pages like iframes, but never had to exclude an entire folder...

    Suggestions/help are warmly appreciated :)
     
    Mr.Dog, Jan 16, 2013 IP
  2. imfusa

    imfusa Active Member

    Messages:
    694
    Likes Received:
    7
    Best Answers:
    0
    Trophy Points:
    70
    #2
    You just add in your robots txt:
     
    imfusa, Jan 16, 2013 IP
    Sharpay likes this.
  3. rakesh.swaransoft

    rakesh.swaransoft Greenhorn

    Messages:
    119
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    18
    #3
    Update your robots.txt file with

    Disallow:/foldername
     
    rakesh.swaransoft, Jan 16, 2013 IP
  4. ryan_uk

    ryan_uk Illustrious Member

    Messages:
    3,983
    Likes Received:
    1,022
    Best Answers:
    33
    Trophy Points:
    465
    #4
    There are bots specifically for images, so you could also disallow them entirely from your site (in case at some point you end up with images outside of /images/ directory).

    User-agent: Googlebot-Image
    Disallow: /
    
    User-agent: MSNBot-Media
    Disallow: /
    
    User-agent: YandexImages
    Disallow: /
    
    User-agent: YandexMedia
    Disallow: /
    
    User-agent: Baiduspider-image
    Disallow: /
    Code (markup):
     
    ryan_uk, Jan 17, 2013 IP
    Sharpay likes this.
  5. Mr.Dog

    Mr.Dog Active Member

    Messages:
    912
    Likes Received:
    18
    Best Answers:
    0
    Trophy Points:
    60
    #5
    OK, so I will disalow 'that foler', but that folder contains a bunch of images.

    How will this affect those images? Will anyone be able to find them via Google Images search? (because a large part of my traffic actually comes from image search)

    Thank you all for helping!
     
    Mr.Dog, Jan 19, 2013 IP
  6. fallen.star

    fallen.star Greenhorn

    Messages:
    140
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    18
    #6
    You should add description with HTML <img src="..." alt="..." /> It also be middle by alt tags while attaching image.
     
    fallen.star, Jan 20, 2013 IP
  7. ryan_uk

    ryan_uk Illustrious Member

    Messages:
    3,983
    Likes Received:
    1,022
    Best Answers:
    33
    Trophy Points:
    465
    #7

    You don't need to disallow anything then if you want the images to show up in image search. Just don't make a link to the images folder or put an index file in there and you'll be fine. The images will still be indexed, but not the folder name as a "web page". You're over-thinking things.
     
    ryan_uk, Feb 2, 2013 IP
  8. Royalbluestuey

    Royalbluestuey Member

    Messages:
    108
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    48
    #8
    You can use wildcards too...this is very useful. I recently had to prevent Google crawling aspx pages in tha folder but allow the html pages to be crawled, very useful.

    Also I don't think anyone has mentioned that Google webmaster tools is great for testing what disallows you have in the robots.txt. I usually use it to make sure that the pages I DO want crawling are still available to the Googlebot.
     
    Royalbluestuey, Feb 11, 2013 IP
  9. MayurKawale

    MayurKawale Well-Known Member

    Messages:
    221
    Likes Received:
    13
    Best Answers:
    0
    Trophy Points:
    170
    #9
    Actually I had same problem. Ya you can disallow these link using Robots.txt but user can still access your folder. So I would suggest you use .htaccess file. In that file use
    which will block all folders to access from smart user. I hope it will helpful to you.
    Thanks
     
    MayurKawale, Mar 29, 2013 IP
    ryan_uk likes this.