I'm running 2 domains on one hosting account. the first one is fine and it's already indexed in google. I've just purchased the second domain 2 days ago, installed wordpress in a new directory in the same hosting I'm using for the first domain. Today I added my new domain in my webmaster tools account but when I tried to submit a sitemap I got this error: URL restricted by robots.txt though I'm sure there's nothing wrong with my robot txt file. any suggestion?
On a hunch.... I believe there is a default setting in wordpress to disallow crawling of wordpress page. It asks if you want your blog to be publicly available. I think it's in the privacy area of the admin. Although i don't think it's a robots.txt file, it is a meta tag Check that first
I edited my wordpress settings and made it publicly available but still get the same error when trying to submit the sitemap. but when I analyze my robot.txt in webmaster tools I find that it hasn't been updated. also written there: should I wait till my robot.txt gets updated?
oh it worked, I just waited till my robot got updated within 24 hours. now I can submit my sitemap without any error
URL Detail Linked From Problem Detected On http://bollywood-indiafmdhamaka.blogspot.com/search/label/Akshay%20Kumar URL restricted by robots.txt unavailable 2/21/09 http://bollywood-indiafmdhamaka.blogspot.com/search/label/Fardeen%20Khan URL restricted by robots.txt unavailable 2/21/09 How can I fix this problem. please help me