I can't say with any certainty that submitting my sitemap yielded Googlebot's recent crawl today, but Google has been on my site all morning/afternoon.
I'd prolly be pretty easy to make your dynamic page generation update your mysite.urls file with all the dynamic urls.....
I'm sure it'd probably would be. I'd love to see one for vbulletin sites. I'm not a programmer, however.
I was wondering the same thing, John. Googlebot crawls my forum site several hundred times a week -- only slightly less often for my main site -- do I need really need any more attention than that? My guess is this might be a way to increase crawling of sites where it isn't already happening, like new sites... where it is, I doubt there's a benefit. Maybe think of it as a "Submit" button for new sites.
For some reason Google doesn't seem too keen on crawling my site all that often (and it's fairly well established going on 4 years now). I submitted my sitemap, and bam, seemed almost like an instant crawl. Again, though, I can't comment one way or another that this was a direct result, so take that for what it's worth.
I have the ultimate test... I have a site that I put online on May 27th. I submitted it to Google, but no results as of yet. Other than a quick "look and see" with a request for a robots.txt file, Googlebot has not been back. Let's see if it gets sucked into the ranks or gets ignored.
Sorry guys, I lied.... I created the file and made the extension .html... Here is my experience: As some of you may know, I have recently updated my computer repairs website . I actually completed the overhaul yesterday. Obviously, Google was not visiting the new urls because I changed all the extensions and how the domains read (I moved from _ to - and .htm to .html + I added more SE friendly names.) Anyways, longer story long, since I submitted the new 'sitemap', Google has indexed every single one of my new pages as a result. This is purely coincidental that they release this tool with my overhaul release, but this is the proof it works... A google search for "Contact NewComputer.ca using the below information" which is text never used on any of my other pages revealed the following: Google Search
Are you saying that you were indexed because of your new site map you submitted using their new site map tool within the last 30 minutes?
Hi all, I just finished putting up an online interface to generate a Google xml sitemap from a simple ls -lR directory listing, you can give it a spin at Create Google xml sitemap from directory listing.
It took a total of 5 hours (from submittal to checking each pages cached copy), but yes, that is what I am saying... Check for yourself: http://216.239.63.104/search?q=cache:http://www.newcomputer.ca/site-map.html Look at that date, I just finished that page yesterday... I had an old site map with a different extension and different name.
Okay! Here is the scoop... I submitted a site that is not even in Google index or anything. I originally submitted it a week or two ago, and although the GoogleBot basically did a "ping" of the main site, it didn't do anything with it. It hasn't been back. I submitted my XML sitemap last night and google, very quickly, did a "ping" of the sitemap file and that was it. It took 3-5 hours before it showed up that it was "OK". Google has yet to make any attempt to index the site looking at the logs. So... submitting a sitemap does not mean instant "indexing" of your site. At least, on my test case, it didn't.
Indeed it would be hard to imagine that it would be otherwise. It's not likely to be like a dog whistle that will instantly summon googlebot to do your bidding. As I think I said earlier, this may be useful for new sites but it's probably best to think of it as a newer better submit button.
So if you do the TEXT file version, how do you inform GOOGLE if you change a pages content. I guess if you add a new page you just add this to the text file. Ian
In my case, it was most likely that my site was indexed previously, and as such, the new sitemap just meant Google needed to comeback for new content, which they have...
I guess, other than the possible "submit" function for new sites, I'm having some difficulty understanding how this is any better than any other site map linked from your home page.