I dunno what to do with Restricted by robots.txt

Discussion in 'Google' started by sat599, Feb 13, 2011.

0
  1. #1
    I have a blog and when i login in to webmaster to increase the traffic i saw Crawl errors and it says Restricted by robots.txt.

    Please let me know what to do with it.
     
    sat599, Feb 13, 2011 IP
  2. saibee

    saibee Peon

    Messages:
    85
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #2
    robots.txt is a script which restricts the search engine crawlers to crawl your website...
    It is basically meant to be used on pages that you don't wat to get indexed or have some personal stuff on...
    Search for the robot script on the html of your blog and delete it if you want your blog to be crawled and hence indexed...

    Best Wishes...
     
    saibee, Feb 13, 2011 IP
  3. milo_pl

    milo_pl Well-Known Member

    Messages:
    215
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    105
    #3
    milo_pl, Feb 13, 2011 IP
  4. DoDo Me

    DoDo Me Peon

    Messages:
    2,257
    Likes Received:
    27
    Best Answers:
    0
    Trophy Points:
    0
    #4
    maybe is because your do not have one, wrong format robots.txt (for example, a HTTP 200 html file) will make your site unindexable
     
    DoDo Me, Feb 13, 2011 IP
  5. bogs

    bogs Active Member

    Messages:
    2,142
    Likes Received:
    16
    Best Answers:
    0
    Trophy Points:
    80
    #5
    bogs, Feb 15, 2011 IP