Robots.txt blocking entire site

Discussion in 'WordPress' started by markdb_sem, Jun 14, 2011.

0
  1. #1
    Hi,

    I recently started using the "Google XML Sitemaps" plugin on my blogs. Now Webmaster tools is telling me that Robots.txt is preventing my sitemaps from being accessed but I have now scaled back my root robots.txt to this http://www.markdb.net/robots.txt

    Has anybody else had this problem or similar and have any solution, i'm pulling my hair out here!
     
    markdb_sem, Jun 14, 2011 IP
  2. lelkoun

    lelkoun Active Member

    Messages:
    288
    Likes Received:
    7
    Best Answers:
    0
    Trophy Points:
    60
    #2
    You should use this instead:

    User-agent: *
Disallow:
    Code (markup):
     
    lelkoun, Jun 14, 2011 IP
  3. markdb_sem

    markdb_sem Peon

    Messages:
    67
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Why's that, lelkoun? Wouldn't that prevent the crawlers?
     
    markdb_sem, Jun 14, 2011 IP
  4. lelkoun

    lelkoun Active Member

    Messages:
    288
    Likes Received:
    7
    Best Answers:
    0
    Trophy Points:
    60
    #4
    lelkoun, Jun 14, 2011 IP
  5. mim24

    mim24 Well-Known Member

    Messages:
    327
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    103
    #5
    not found.
     
    mim24, Jun 15, 2011 IP