Shared host blocking Googlebot?

Discussion in 'Site & Server Administration' started by Kerosene, Dec 27, 2011.

  1. #1
    One of the domains on my shared hosting account is inaccessible to Googlebot.
    But my other domains, on the same shared IP are still being crawled without any problems.
    ---
    Using Google Webmaster Tools on the "problem" domain...
    Fetch as Googlebot brings up "Missing robots.txt".
    And every crawl attempt returns "robots.txt unreachable".
    The logs show no record of Googlebot accessing the site.
    ---
    I moved the domain to a dedicated IP this morning, and the problem went away immediately.
    What was causing Googlebot to be blocked on that ONE particular domain, but not others on the same IP?
     
    Kerosene, Dec 27, 2011 IP
  2. @nibb

    @nibb Greenhorn

    Messages:
    74
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    16
    #2
    If the other websites using the same shared IP can be accessed it means it has nothing to do with the host. Its a problem with your website.

    Maybe it was having a timeout.
     
    @nibb, Jan 11, 2012 IP
  3. Kerosene

    Kerosene Alpha & Omega™ Staff

    Messages:
    11,366
    Likes Received:
    575
    Best Answers:
    4
    Trophy Points:
    385
    #3
    On the same server/IP?
     
    Kerosene, Jan 12, 2012 IP