Where exactly place robots.txt

Discussion in 'Search Engine Optimization' started by infoaddicts, Mar 2, 2009.

0
  1. #1
    hi

    which is ther ight area on web host to place robots.txt

    inside Public_html or outside it?
     
    infoaddicts, Mar 2, 2009 IP
  2. kailash

    kailash Well-Known Member

    Messages:
    1,248
    Likes Received:
    42
    Best Answers:
    0
    Trophy Points:
    190
    #2
    You will have to put it inside public_html folder otherwise search engine spiders will not be able to access your robots.txt file.

    Kailash
     
    kailash, Mar 2, 2009 IP
  3. infoaddicts

    infoaddicts Active Member

    Messages:
    397
    Likes Received:
    7
    Best Answers:
    0
    Trophy Points:
    60
    #3
    kailash, problem is i dont want some of the folder on webhost to be indexed by spiders..accordingly have added disallow in robots .txt but ...still see that folder indexed by google..

    a nu suggestion
     
    infoaddicts, Mar 2, 2009 IP
  4. kailash

    kailash Well-Known Member

    Messages:
    1,248
    Likes Received:
    42
    Best Answers:
    0
    Trophy Points:
    190
    #4
    It may possible that they have been indexed before you put robots.txt. You can also block the URLs or folders from your Google Webmaster tools. Login to your webmaster account and then go to Tools -> Remove URLs

    Kailash
     
    kailash, Mar 2, 2009 IP
  5. DelhiCarWash

    DelhiCarWash Well-Known Member

    Messages:
    184
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    105
    #5
    Place it inside "public_html" folder
     
    DelhiCarWash, Mar 2, 2009 IP
  6. tysonboxing

    tysonboxing Guest

    Messages:
    11
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #6
    if the robot doesnt do what robots.txt says, you need to block the robot with a .htaccess!
     
    tysonboxing, Mar 2, 2009 IP