Can't fetch as Googlebot

Discussion in 'Google' started by exlvl, Oct 9, 2010.

  1. #1
    [​IMG]

    Firstly I used

    User-agent: *
    Disllow: /
    
    Code (markup):
    But then I have changed it to (about 8 hours ago)

    User-agent: *
    Allow: /
    Code (markup):
    Now have no restrictions for bots in my robots.txt
    http://www.forumese.com/robots.txt

    But its saying "Denied by robots.txt" in webmasters account.
    and thats why my pages are not being indexed from the sitemap I submitted.
    Is it normal to take that much time for changes to take effect?
     
    exlvl, Oct 9, 2010 IP
  2. Abhik

    Abhik ..:: The ONE ::..

    Messages:
    11,337
    Likes Received:
    606
    Best Answers:
    0
    Trophy Points:
    410
    Digital Goods:
    2
    #2
    Check when your robots.txt last checked by Googlebot..
     
    Abhik, Oct 9, 2010 IP
  3. exlvl

    exlvl Peon

    Messages:
    51
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    its 15 hours ago, and thats my question, does it take that much time for changes to take effect?
     
    exlvl, Oct 9, 2010 IP
  4. longcall911

    longcall911 Peon

    Messages:
    1,672
    Likes Received:
    87
    Best Answers:
    0
    Trophy Points:
    0
    #4
    You have a major error!

    Do not put "/" after disallow. That says 'disallow all pages in the root folder'.

    Do not use allow. There is no such thing.

    It must appear as:

    User-agent: *
    Disallow: 
    Code (markup):
    If nothing follows the disallow tag, all are allowed.
     
    longcall911, Oct 9, 2010 IP
  5. Digital Future

    Digital Future Peon

    Messages:
    23
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #5
    digitech is right, you have change your robots.txt 8 hours ago but it hasn't downloaded the file for 15 hours.
     
    Digital Future, Oct 9, 2010 IP
  6. AirForce1

    AirForce1 Peon

    Messages:
    1,325
    Likes Received:
    13
    Best Answers:
    0
    Trophy Points:
    0
    #6
    Hi,

    I think cleaning the cache on your site might be helpful after such a change.

    all the best,
     
    AirForce1, Oct 10, 2010 IP
  7. longcall911

    longcall911 Peon

    Messages:
    1,672
    Likes Received:
    87
    Best Answers:
    0
    Trophy Points:
    0
    #7
    I repeat:

    User-agent: *
    Disallow:


    Your current file is wrong!!!
     
    longcall911, Oct 10, 2010 IP
  8. sweta.singh.98

    sweta.singh.98 Guest

    Messages:
    628
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #8
    Now its Correct .. Enjoy :)
     
    sweta.singh.98, Oct 10, 2010 IP