Duplicate entries in XML Sitemap

Discussion in 'Google Sitemaps' started by ColeenB, Feb 26, 2009.

  1. #1
    I've run an XML sitemap on my site, and it's listing some duplicate (or nearly duplicate) pages. For example, on each product page, there's a link saying "Email A Friend". When you click that, it takes you to a page where you can fill out email info. The product information is also displayed on this page.

    On the XML sitemap I'm getting one page like:

    http://www.mysite.com/product1

    And another like:

    http://www.mysite.com/emailafriend/product1

    I definitely want my product page to be indexed, but I don't know that having the email a friend page indexed adds anything. Should I exclude these duplicates from my sitemap?
     
    ColeenB, Feb 26, 2009 IP
  2. shailendra

    shailendra Peon

    Messages:
    1,225
    Likes Received:
    18
    Best Answers:
    0
    Trophy Points:
    0
    #2
    you can remove email a friend URL from sitemap or use robots.txt to block the access to it
     
    shailendra, Feb 26, 2009 IP
  3. websitetools

    websitetools Well-Known Member

    Messages:
    1,513
    Likes Received:
    25
    Best Answers:
    4
    Trophy Points:
    170
    #3
    Why not eother use noindex, robots.txt or configure sitemap crawler list filters. E.g. A1 sitemap Generator supports those, but other capable sitemappers should as well.
     
    websitetools, Feb 27, 2009 IP
  4. ColeenB

    ColeenB Peon

    Messages:
    123
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Thanks. I should probably use robots.txt so that crawlers won't waste their time there either.
     
    ColeenB, Feb 27, 2009 IP