How to stop GoogleBot accessing a file

Discussion in 'Google' started by photosales.co.nz, Jul 17, 2007.

  1. #1
    Is it possible to ban GoogleBot and other robots from accessing a paticular file?

    I do not want it to be able to access my /download.php file, as I just saw in my logs it somehow manages to access that file, and download a picture, which would not be good for that to end up in google's index...
     
    photosales.co.nz, Jul 17, 2007 IP
  2. cormac

    cormac Peon

    Messages:
    3,662
    Likes Received:
    222
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Use the robots.txt file to block the bots. See Google's help on how to implement the rules into the file.
     
    cormac, Jul 17, 2007 IP
  3. gemini181

    gemini181 Well-Known Member

    Messages:
    2,883
    Likes Received:
    134
    Best Answers:
    0
    Trophy Points:
    155
    #3
    I wanted to learn this soon anyway :)

    "For instance, to block Googlebot entirely, you can use the following syntax:

    User-agent: Googlebot
    Disallow: /

    Allowing Googlebot
    If you want to block access to all bots other than the Googlebot, you can use the following syntax:

    User-agent: *
    Disallow: /

    User-agent: Googlebot
    Disallow:

    Googlebot follows the line directed at it, rather than the line directed at everyone...."
     
    gemini181, Jul 17, 2007 IP
  4. photosales.co.nz

    photosales.co.nz Peon

    Messages:
    371
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    0
    #4
    So here is my robots.txt file at the moment:

    User-agent: *
    Disallow: /cgi-bin/
    Disallow: /digital/
    Disallow: /files/
    Disallow: /ftp/
    Disallow: /gal_images/
    Disallow: /images/
    Disallow: /js/
    Disallow: /logo/
    Disallow: /manager/
    Disallow: /sample_videos/
    Disallow: /stock_photos/
    Disallow: /stock_videos/
    Disallow: /styles/
    Disallow: /temp/
    Disallow: /uploaded_files/
    Disallow: /uploaded_images/
    Disallow: /lang/
    Disallow: /resources/

    Sitemap: http://www.photosales.co.nz/sitemap.php

    If I just add:
    Disallow: download.php
    Disallow: money_return.php

    Will that stop it accessing those files?>
     
    photosales.co.nz, Jul 17, 2007 IP
  5. Vadym

    Vadym Peon

    Messages:
    2,857
    Likes Received:
    95
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Er, you could make a verification pop-up or something of the sort. Just ask to verify some picture, "to make sure the person is real" or whatever.
     
    Vadym, Jul 17, 2007 IP
  6. trichnosis

    trichnosis Prominent Member

    Messages:
    13,785
    Likes Received:
    333
    Best Answers:
    0
    Trophy Points:
    300
    #6
    the best way is robots.txt file but you can also use meta tags as an alternative way
     
    trichnosis, Jul 17, 2007 IP
  7. godmode

    godmode Well-Known Member

    Messages:
    4,453
    Likes Received:
    156
    Best Answers:
    0
    Trophy Points:
    190
    #7
    just add

    disallow: /download.php

    to robot.txt
     
    godmode, Jul 17, 2007 IP