1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Google Sitemaps FAQ

Discussion in 'Google Sitemaps' started by astup1didiot, Oct 27, 2007.

  1. shawnden

    shawnden Peon

    Messages:
    54
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #61
    I thought that you stated if it works for the robots I should be OK. Navigation on my site is simple from a human perscpective. Don't think I really need a sitemap other then to help google. However, it didnt seem to index but 2 of my pages. Thanks for the reply.
    SEMrush
     
    shawnden, Feb 12, 2008 IP
    SEMrush
  2. astup1didiot

    astup1didiot Notable Member

    Messages:
    5,927
    Likes Received:
    269
    Best Answers:
    0
    Trophy Points:
    280
    #62
    Sitemaps are used as a "guide" for search engine crawlers, it doesn't mean all your pages will get indexed. It's a source for the crawlers to find a list of all your valid web pages.
     
    astup1didiot, Feb 15, 2008 IP
  3. skaterkee

    skaterkee Well-Known Member

    Messages:
    651
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    138
    #63
    If I don't list all my pages in a sitemap, do those pages still get crawled? I use a php news script so it is not feasible to list all the news. But I can list all my categroies etc?
     
    skaterkee, Feb 20, 2008 IP
  4. KVeil

    KVeil Member

    Messages:
    79
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    41
    #64
    shawnden and skaterkee: as ssandecki said, a sitemap helps the Google spider to find all the pages on you web site, especially those which may not be linked - accidentally or deliberately - within your site. However, a sitemap does NOT guarantee that all the pages listed in it will be indexed.

    So by putting up a complete sitemap, your are "helping" you site being completely indexed by Google (Yahoo and MS Live also use the sitemap).


    K
     
    KVeil, Feb 20, 2008 IP
  5. astup1didiot

    astup1didiot Notable Member

    Messages:
    5,927
    Likes Received:
    269
    Best Answers:
    0
    Trophy Points:
    280
    #65
    Yes, you don't need to list all the URLs. Google will still find them while crawling your site, unless you use robots.txt to block those specific URLs from indexing\crawling.
     
    astup1didiot, Feb 20, 2008 IP
  6. Essee Yo

    Essee Yo Peon

    Messages:
    7
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #66
    Hello...we're trying to locate a decent sitemap generator, but have had no luck after demo-ing about a dozen. You had listed the tool from AuditMySite.com, which seemed to do the best job of crawling. However, my browser window closes itself when the tool is finished. Besides, when I attempted to use this tool a few months back I was able to export a sitemap.xml...but the resulting file had obscure XML tags. According to Google, the correct format uses the <urlset> and <loc>, etc. tags. This particular file used a different notation all together.

    Anyway, we've tried installing tools on our server, but things just don't "cooperate". We also tried to use the CoffeeCup software...this seemed to work well, except it only indexed 414 pages out of thousands on our site. I know it isn't imperative to have every page included, but that number just seemed rather low to me.

    At this point I'm getting frustrated trying to find something that actually works :(

    Thanks!
     
    Essee Yo, Feb 20, 2008 IP
  7. astup1didiot

    astup1didiot Notable Member

    Messages:
    5,927
    Likes Received:
    269
    Best Answers:
    0
    Trophy Points:
    280
    #67
    Did you try the Vgios one I listed?
     
    astup1didiot, Feb 20, 2008 IP
  8. SFNUM8

    SFNUM8 Active Member

    Messages:
    297
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    58
    #68
    thanks for the info, I used freesitemapgenerator.com it took a couple of days for them to crawl my site but it worked great.
     
    SFNUM8, Feb 21, 2008 IP
  9. astup1didiot

    astup1didiot Notable Member

    Messages:
    5,927
    Likes Received:
    269
    Best Answers:
    0
    Trophy Points:
    280
    #69
    Glad to hear, i'm going to go through all the pages in this thread in a few days and update more sitemap generating websites and add more content to the FAQ.
     
    astup1didiot, Feb 21, 2008 IP
  10. MOG

    MOG Well-Known Member

    Messages:
    2,219
    Likes Received:
    163
    Best Answers:
    0
    Trophy Points:
    180
    #70
    Great FAQ, I have been using the www.auditmypc one for ages now and IMHO its the best there is online. Fast, reliable and great with big sites...
     
    MOG, Feb 22, 2008 IP
    usasportstraining likes this.
  11. astup1didiot

    astup1didiot Notable Member

    Messages:
    5,927
    Likes Received:
    269
    Best Answers:
    0
    Trophy Points:
    280
    #71
    I'm a fan of Vgios's downloadable generator, works great! :)
     
    astup1didiot, Feb 22, 2008 IP
  12. websitetools

    websitetools Well-Known Member

    Messages:
    1,510
    Likes Received:
    25
    Best Answers:
    4
    Trophy Points:
    170
    #72
    Have you tried A1 Sitemap Generator ?
     
    websitetools, Feb 23, 2008 IP
  13. KVeil

    KVeil Member

    Messages:
    79
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    41
    #73
    KVeil, Feb 24, 2008 IP
  14. dipali

    dipali Peon

    Messages:
    450
    Likes Received:
    9
    Best Answers:
    0
    Trophy Points:
    0
    #74
    Hello All,


    I have been Submitted sitemaps "garment.fibre2fashion.com/garment.xml" it contains 15000 url but in webmaster toll it will shows me only 5000 url are indexed.


    please help me out to resolve this error?
    Reply With Quote

    Txs in Advance
    Dipali
     
    dipali, Feb 27, 2008 IP
  15. astup1didiot

    astup1didiot Notable Member

    Messages:
    5,927
    Likes Received:
    269
    Best Answers:
    0
    Trophy Points:
    280
    #75
    It's recommended to only use 5,000 URLs per sitemap. I actually have multiple sitemaps for my website, I split them into 2,500 URLs.
     
    astup1didiot, Feb 27, 2008 IP
  16. PaddyL

    PaddyL Active Member

    Messages:
    69
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    50
    #76
    All this discussion is great for explaining how to create and use a sitemap (I have used one for a long time now).

    However...

    If all your pages are reachable starting from your home page, is there actually any point? In other words, unless you have pages that a crawler wouldn't otherwise find, what difference does a sitemap make?
     
    PaddyL, Feb 29, 2008 IP
  17. astup1didiot

    astup1didiot Notable Member

    Messages:
    5,927
    Likes Received:
    269
    Best Answers:
    0
    Trophy Points:
    280
    #77
    This is a great point, usually after a website has been crawled and indexed for a good period and has a good amount of backlinks a sitemap is really not needed, but new upcoming sites should always use one.
     
    astup1didiot, Feb 29, 2008 IP
  18. catanich

    catanich Peon

    Messages:
    1,921
    Likes Received:
    40
    Best Answers:
    0
    Trophy Points:
    0
    #78
    PaddyL You are correct as far as you comment goes, but by adding a site map, you shorten the time for google to obtain all of your url.

    I use xml site maps with ~35,000 URLs in them, and use Webmaster Tools to submit them. Within weeks, most of the URLs have been indexed.
     
    catanich, Mar 7, 2008 IP
  19. TruthTrekker

    TruthTrekker Guest

    Messages:
    58
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #79
    2 questions:

    A) I created an XML sitemap and succussfully submitted it to Google. Should I still create a sitemap link on my site to the XML page?

    B) I plan on using a PHP jump script as I add more products and content to my site. How can I use a robot.txt file so Google won't index the folder containing the PHP script?
     
    TruthTrekker, Mar 7, 2008 IP
  20. PaddyL

    PaddyL Active Member

    Messages:
    69
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    50
    #80
    A) You can either create a link on your home page, or, more usefully, submit directly to Google.

    http://www.google.com/webmasters/

    B) Your robots.txt should be in your website's root directory. The line you want is

    User-agent: *
    Disallow: /php-scripts/
    Code (markup):
    where "php-scripts" would be the path to the scripts.

    Find more information at
    http://www.robotstxt.org/
     
    PaddyL, Mar 8, 2008 IP
    usasportstraining likes this.