Has anyone used the Google Beta SiteMap Generator https://www.google.com/webmasters/sitemaps/docs/en/sitemap-generator.html? This appears to be a great way to get large sites indexed properly by Google, but curious if anyone has any negative or positive experiences with the Beta tool? Are there any unintended consequences to submitting your site to Google in this fashion? Enquiring minds ...
I would say the biggest unintended consequence is telling Google (and anyone else who knows the URL to the XML sitemap) about documents on your web space that were previously unknown. Security through Obscurity doesn't work when a local script is walking through directory trees and outputting everything...
I'm not saying that's how you *should* use the web, but many folks use their web space as second hard drive and such... p.s. good times: http://www.google.com/search?q=allinurl:sitemap.xml.gz
I used a hacked apart version of it for my oscommerce site - on Friday I had 897 pages indexed by Google despaite thousands of backlinks and 4 years online with frequent google visits. Today I have 8,954 pages indexed. I submitted my sitemap yesterday morning - thats what I call a result! I would strongly recommend it, also recommend putting a link to your sitemap somewhere on your homepage for other robots - it really helps them get around your site.
I think the two things are totally unrelated. There have been reports elsewhere on this forum of wild swings in the number of pages indexed, by Google in the last 24 to 48 hours. see http://forums.digitalpoint.com/showpost.php?p=206395&postcount=1995 There is a difference between coincidence and cause and effect.
Interesting. I have a dynamic site, so google's generator wasn't an option I just generate my own xml and submit it to google. As for indexing unwanted info - doesn't it follow robots.txt rules? At least you can hide the xml - https://www.google.com/webmasters/sitemaps/docs/en/faq.html#s6
You are right debare. Writing a spider is challenging. It has to account for every level of quality of html out in the wild on the internet. The GoogleBot programmers probably said, "I've had it! Let the webmasters and site owner tell us what to index." And hence, Google sitemaps was born. It is great for dynamic sites. Saves bandwidth and indexing time. It works best for dynamic sites. Static sites are mostly well indexed.
I made a sitemap file and placed it on a small site. I put a page that is not linked from anywhere else and placed a alert email in it to check when googlebot comes. Nothing so far ...
Sitemap is truely a best of getting the pages indexed on Google. I have tried it and happy with the outcome.