1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Google wont accept my sitemap

Discussion in 'Google Sitemaps' started by hawksrnm1, Nov 10, 2008.

  1. #1
    I have uploaded sitemaps for my site patriotsrush.com and few others but when I checked my google webmaster account it gave me the following error message.
    SEMrush
    We were unable to crawl your Sitemap because we found a robots.txt file at the root of your site but were unable to download it. Please ensure that it is accessible or remove it completely.

    Im not sure why this is because I haven't uploaded a robots.txt file into any of the domains on my server and I checked to make sure there was nothing in there and there wasn't. What do I do to fix this? Until I can get it uploaded right google wont index the sites. Thanks a lot guys.

    Sean
     
    hawksrnm1, Nov 10, 2008 IP
    SEMrush
  2. gbh1935

    gbh1935 Peon

    Messages:
    585
    Likes Received:
    11
    Best Answers:
    0
    Trophy Points:
    0
    #2
    When you try to download the robots.txt it returns a 500 error. Are you using a hosting service or do you have your own server. You should check your web server config and if apache .htaccess to see if you have something hardcoded to handle request for robots.txt
     
    gbh1935, Nov 10, 2008 IP
  3. hawksrnm1

    hawksrnm1 Peon

    Messages:
    371
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    0
    #3
    I haven't even used a robots.txt. Thats why it is confused. Google is saying they cant open it. I am new to a lot of this but isnt a robots.txt file for pages you dont want google to see? I have a Virtual Private Server.
     
    hawksrnm1, Nov 11, 2008 IP
  4. websitetools

    websitetools Well-Known Member

    Messages:
    1,510
    Likes Received:
    25
    Best Answers:
    4
    Trophy Points:
    170
    #4
    It does sound like a somewhat odd error. What's your website URL?
     
    websitetools, Nov 11, 2008 IP
  5. hawksrnm1

    hawksrnm1 Peon

    Messages:
    371
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    0
    #5
    one of them is patriotsrush.com but this is happening all of the sites in the network that I have completed so far. This was the first time I have used this sitemap creator could that be the problem?
     
    hawksrnm1, Nov 11, 2008 IP
  6. websitetools

    websitetools Well-Known Member

    Messages:
    1,510
    Likes Received:
    25
    Best Answers:
    4
    Trophy Points:
    170
    #6
    I don't know what sitemap generator you are using? Are you using A1 Sitemap Generator?


    Anyways, I checked a few things quickly.

    1)
    If you try check
    http://www.patriotsrush.com/robots.txt
    You will see it's giving an internal server error (500)

    Google's error message to you is misleading. They should simply have stated your webserver was throwing error response 500 when requesting the URL.

    Internal server errors are often caused by Apache .htaccess configuration errors (e.g. in mod_rewrite but it can be anything). you should check your webserver log!

    2)
    By the way, I also noticed your website can be accessed from both
    http://patriotsrush.com/ and http://www.patriotsrush.com/ (you don't do a redirect)
     
    websitetools, Nov 11, 2008 IP
  7. hawksrnm1

    hawksrnm1 Peon

    Messages:
    371
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    0
    #7
    I am using igoomap...i am using MAC OS
     
    hawksrnm1, Nov 11, 2008 IP
  8. hawksrnm1

    hawksrnm1 Peon

    Messages:
    371
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    0
    #8
    OK I tried creating a blank robots.txt file and uploading it to my server and then resubmitted my sitemap but its still coming up with the error. Please someone help me out.
     
    hawksrnm1, Nov 11, 2008 IP
  9. gbh1935

    gbh1935 Peon

    Messages:
    585
    Likes Received:
    11
    Best Answers:
    0
    Trophy Points:
    0
    #9
    what is the web server you are using? Apache, IIS, ???
     
    gbh1935, Nov 11, 2008 IP
  10. penyu

    penyu Peon

    Messages:
    96
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #10
    i think i have same probelm like hawksrnm1, wait for more solution :D
     
    penyu, Nov 11, 2008 IP
  11. websitetools

    websitetools Well-Known Member

    Messages:
    1,510
    Likes Received:
    25
    Best Answers:
    4
    Trophy Points:
    170
    #11
    It's still odd that your webserver responded with error 500 before... But nontheless, I suggest you now wait a couple of days to see if Google's error goes away. It may take a little time for Google to update their status.
     
    websitetools, Nov 12, 2008 IP
  12. vovaNux

    vovaNux Peon

    Messages:
    47
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #12
    Did you try to create robots.txt with link to your sitemap file?
     
    vovaNux, Nov 12, 2008 IP
  13. hawksrnm1

    hawksrnm1 Peon

    Messages:
    371
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    0
    #13
    It is a VPS with Liquid Web
     
    hawksrnm1, Nov 12, 2008 IP
  14. hawksrnm1

    hawksrnm1 Peon

    Messages:
    371
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    0
    #14

    That is a good idea, just make the robots.txt point to allow the sitemap.
     
    hawksrnm1, Nov 12, 2008 IP
  15. hawksrnm1

    hawksrnm1 Peon

    Messages:
    371
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    0
    #15

    Just wanted to let you know this has been resolved. I added an allow to the robots.txt for my sitemap and it has not picked it up. Thanks for your help.
     
    hawksrnm1, Nov 12, 2008 IP
  16. misteranss

    misteranss Peon

    Messages:
    27
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #16
    Yes This is right
     
    misteranss, Nov 13, 2008 IP
  17. pmcross

    pmcross Peon

    Messages:
    95
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #17
    Yep, the robots file looks good to me, i had a similar problem as well
     
    pmcross, Nov 13, 2008 IP
  18. ciplseo

    ciplseo Peon

    Messages:
    338
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    0
    #18
    With the intention is an excellent suggestion, immediately formulate the robots.txt position to allocate the sitemap.
     
    ciplseo, Nov 14, 2008 IP
  19. jkookserv

    jkookserv Peon

    Messages:
    77
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #19
    You can use robots.txt generator.
     
    jkookserv, Nov 14, 2008 IP
  20. tanenqq

    tanenqq Well-Known Member

    Messages:
    124
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    101
    #20
    I have submit and donot see any error
     
    tanenqq, Nov 14, 2008 IP