URLs restricted by robots.txt..... ?

Discussion in 'Google Sitemaps' started by jamie01, Mar 19, 2007.

  1. #1
    HI

    What it means in Google sitemap.... my website has 60 errors in this column..
    Web crawl errors... like :
    HTTP errors
    0
    --
    Not found
    0
    --
    URLs not followed
    0
    --
    URLs restricted by robots.txt
    60

    Details »
    URLs timed out
    0
    --
    Unreachable URLs
    0


    Please explain me if it has any impact on SERP.......and which URL restricted by robots?

    Thanks
     
    jamie01, Mar 19, 2007 IP
  2. Mwb2k

    Mwb2k Peon

    Messages:
    92
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Uhm, you can define the contents of robots.txt yourself, so how can we know? :)
     
    Mwb2k, Mar 19, 2007 IP
  3. abcde

    abcde Well-Known Member

    Messages:
    151
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    108
    #3
    I guess its this site ?? --> ayhu.com

    take a look at http://www.ayhu.com/robots.txt

    User-agent: *
    Disallow: admin/
    Disallow: includes/
    Disallow: templates/
    Disallow: logout.php
    Disallow: suggest-link.php
    Disallow: suggest-category.php
     
    abcde, Mar 19, 2007 IP
  4. jamie01

    jamie01 Guest

    Messages:
    84
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    So does google sitemap shows those URL which has been disallowed in robort.txt ? I am worried if this due to other Problems....pls let me know how to get ride from this ?

    Thanks
     
    jamie01, Mar 19, 2007 IP
  5. jamie01

    jamie01 Guest

    Messages:
    84
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #5

    No, sorry...not this one
     
    jamie01, Mar 19, 2007 IP
  6. abcde

    abcde Well-Known Member

    Messages:
    151
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    108
    #6
    delete the robots.txt and allow to crawl the whole site. BUT ist not a problem or an error so theres nothing you can do.
     
    abcde, Mar 19, 2007 IP
  7. daykevi

    daykevi Peon

    Messages:
    106
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #7
    No not at all, if you restricted some url using the robots.txt file it is not going to hurt u.
     
    daykevi, Mar 19, 2007 IP
  8. jamie01

    jamie01 Guest

    Messages:
    84
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #8
    HI abcde

    Is it a hurt for me ? would it be better than now if the robot.txt deleted ?

    Thanks
     
    jamie01, Mar 19, 2007 IP
  9. abcde

    abcde Well-Known Member

    Messages:
    151
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    108
    #9
    abcde, Mar 19, 2007 IP
  10. jamie01

    jamie01 Guest

    Messages:
    84
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #10
    jamie01, Mar 19, 2007 IP