Sitemap contains urls which are blocked by robots.txt.

Discussion in 'Search Engine Optimization' started by neha.pandey, Dec 11, 2014.

0
  1. #1
    Hi I have hosted new site and i am getting this error on Google Webmasters tool That"Sitemap contains urls which are blocked by robots.txt."
    Due to this i m unable to index my site properly on the google.

    Sitemap URL is http://clusterevents.com/sitemap.xml
    Robot.txt at http://clusterevents.com/robots.txt

    I have done many time submitting sitemap to Google Webmasters tool but i don't know why i m getting this message as my robot.txt file is not blocking any pages.

    Please help me out
     

    Attached Files:

    neha.pandey, Dec 11, 2014 IP
  2. Ratty

    Ratty Active Member

    Messages:
    565
    Likes Received:
    12
    Best Answers:
    0
    Trophy Points:
    95
    Digital Goods:
    1
    #2
    Try going into the Google Webmasters tool and manually crawl the page with the 'Fetch as Google' tool and see if you still get the same message.
     
    Ratty, Dec 11, 2014 IP
  3. mnmani

    mnmani Well-Known Member

    Messages:
    379
    Likes Received:
    11
    Best Answers:
    6
    Trophy Points:
    195
    #3
    robots.txt should also include Allow directive like below
    
User-agent: *
Disallow: /wp-admin/
Allow: /
Sitemap: http://clusterevents.com/sitemap.xml
    Code (markup):
    and sitemap.xml is not in correct format. you have to rectify that.
     
    mnmani, Dec 11, 2014 IP
  4. braulio

    braulio Active Member

    Messages:
    70
    Likes Received:
    7
    Best Answers:
    1
    Trophy Points:
    95
    #4
    Create your sitemap using the tool @ the site below. This is how we create all our sitemap for our customers.
    https://www.xml-sitemaps.com/

    Also, below is google's support literature concerning robots.txt. Please make sure and use the robots.txt test @ Google Webmaster Tools site.
    https://support.google.com/webmasters/answer/6062608?hl=en&ref_topic=6061961

    Robots.txt tester instructions are below.
    https://support.google.com/webmasters/answer/6062598

    Please let us know if the above information helped you.

    Good luck, Braulio
     
    braulio, Dec 11, 2014 IP
  5. neha.pandey

    neha.pandey Member

    Messages:
    43
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    38
    #5
    Thank you all helping me out :) kisses and hugs to all
     
    neha.pandey, Dec 20, 2014 IP
  6. neha.pandey

    neha.pandey Member

    Messages:
    43
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    38
    #6
    now i m getting another error like
    "Some URLs listed in this Sitemap have a high response time. This may indicate a problem with your server or with the content of the page."
    earlier it was 0.5 and i reduce it to 0.1 .

    But i have no clue i m doing it right or not . please let me know what to do ahead to make google accept my sitemap completely
     
    neha.pandey, Dec 20, 2014 IP