URLs restricted by robots.txt (i've deleted robots.txt)

Discussion in 'Google Sitemaps' started by mr_vampire, Jan 22, 2010.

  1. #1
    The webmaster tool says URL restricted by robots.txt.

    Right now I don't have any file robots.txt file .

    How is it possible ?
    Also Google is not indexing my site, though I've done all the conventional stuffs. :confused:
     
    mr_vampire, Jan 22, 2010 IP
  2. 9daywebsites

    9daywebsites Peon

    Messages:
    10
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #2
    might be you have dis-allowed that url in your robots.txt.

    first create robots.txt file which is necessary and atlast in your file insert the sitemap.xml url such as

    Sitemap: http://www.domainname.com/sitemap.xml

    but before that create and place your sitemap.xml

    thank you
     
    9daywebsites, Jan 23, 2010 IP
  3. Las vegas Home Rentals

    Las vegas Home Rentals Peon

    Messages:
    10
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #3
    robots.txt file is used to hide or dis-allow some pages to be crawled by the search engines. for e.g.:- your pages after login, adninistrator pages and images you have kept in root folder etc.

    so insert it in your root folder.
     
    Las vegas Home Rentals, Jan 23, 2010 IP
  4. mr_vampire

    mr_vampire Well-Known Member

    Messages:
    458
    Likes Received:
    9
    Best Answers:
    0
    Trophy Points:
    130
    Articles:
    1
    #4
    I've already done .. still no progress . My other sites with same setting are indexed regularly. This thing has got into my skins :D
     
    mr_vampire, Jan 23, 2010 IP
  5. Lex350

    Lex350 Notable Member

    Messages:
    1,889
    Likes Received:
    31
    Best Answers:
    0
    Trophy Points:
    215
    #5
    You dont need to delete your robots file, infact you dont want to do that at all. You should copy paste or link your file so we can look at it.
     
    Lex350, Jan 24, 2010 IP