Best robots.txt format for my new blog?

Discussion in 'Search Engine Optimization' started by geekology, Dec 22, 2007.

0
  1. #1
    HI Everyone,

    Can anyone suggest the best considered robots.txt format for my wordpress hosted blog? Or a link to the same?
     
    geekology, Dec 22, 2007 IP
  2. stOx

    stOx Notable Member

    Messages:
    6,426
    Likes Received:
    130
    Best Answers:
    0
    Trophy Points:
    230
    #2
    depends what pages you want to resrict access to.
     
    stOx, Dec 22, 2007 IP
  3. geekology

    geekology Well-Known Member

    Messages:
    545
    Likes Received:
    15
    Best Answers:
    0
    Trophy Points:
    110
    #3
    @ STOX: I am asking this for a very new blog...or if i can get some good examples?
     
    geekology, Dec 23, 2007 IP
  4. Kaizoku

    Kaizoku Well-Known Member

    Messages:
    1,261
    Likes Received:
    20
    Best Answers:
    1
    Trophy Points:
    105
    #4
    Sounds like you don't know what robots.txt does...
    User-agent: *
    Disallow: /folders-you-don't-wan't-se-to-index/
    Disallow: /admin/
     
    Kaizoku, Dec 23, 2007 IP