robots.txt in each folder

Discussion in 'Search Engine Optimization' started by blue_ocean, Jul 11, 2008.

0
  1. #1
    Can i make a diff. robots.txt file in each diff folder? Will it work?
     
    blue_ocean, Jul 11, 2008 IP
  2. oseymour

    oseymour Well-Known Member

    Messages:
    3,960
    Likes Received:
    92
    Best Answers:
    0
    Trophy Points:
    135
    #2
    why do you want to do that? one in the main folder does the same thing....and I think the spiders only look for the robots file in the root directory anyway
     
    oseymour, Jul 11, 2008 IP
  3. webcosmo

    webcosmo Notable Member

    Messages:
    5,840
    Likes Received:
    153
    Best Answers:
    2
    Trophy Points:
    255
    #3
    It will work, but thats totally redundant; just put it in the root folder.
     
    webcosmo, Jul 11, 2008 IP
  4. blue_ocean

    blue_ocean Active Member

    Messages:
    359
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    75
    #4
    i've one in the root. i'm using a script in folder which comes with robots.txt. so i have to keep robots.txt file there also otherwise i've to copy it to the root by changing all its urls.

    I checked it out and came to know robots.txt in folder also works as fine as roots........
     
    blue_ocean, Jul 11, 2008 IP