robots.txt files with similar names

Discussion in 'robots.txt' started by tutordelphia, Nov 1, 2012.

  1. #1
    I created a tutor sign in on my site which I didn't want indexed in a folder called \tutor

    I had another folder which listed my tutors called \tutors

    I tried to block the tutor folder with Disallow: /tutor but everything in my tutors folder ended up being disallowed as well (and it unfortunately took me some time to realize it).

    I have two questions. I know I had too similar names, but why did google block tutors when I had written tutor?

    The second question is when will google find the descriptions for these pages? It has been a week since I fixed it. I checked that google can crawl the pages now but the description still says it is blocked in google's SERP.
     
    tutordelphia, Nov 1, 2012 IP
  2. adwordmedia

    adwordmedia Greenhorn

    Messages:
    27
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    11
    #2
    It depends on Google Algorithm. In some cases, they block with same named folder and and may be not. If your pages are cached then your description of pages must be cached in Google SERP Tool.

    So make different rules if you want to block two different folders.

    Thanks
    Sean
     
    adwordmedia, Nov 5, 2012 IP