Dynamic sitemaps?

Hi folks, Google has been crawling my site for a while now, but for some reason, only my index page is being indexed, and my deeper pages are not.

I think its about time I got involved with the scary google sitemaps.

Now, my entire website is database driven, and the pages change in content frequently, and the structure of my internal links is also changing frequently. (for example, my index page has links to the 10 most recent pages within my site, and this list of links changes frequently).

From what I've learned, there are several ways of creating a site map. The first is to manually enter the details of each of your pages, and the second is to use a sitemap generator which crawls your site and adss the details to the sitemap itself.

But what if the sitemap file itself was a php script.
Can we not just get a file called sitemap.xml and use a .htaccess directive to treat it as a php script.

Then, when this .xml script is requested via google, it connects to my database, and generates and returns dynamically a XML output of URLS that are contained within my site?

I suspect people are probably doing this allready. How has this worked out for you? Is there a problem with the fact that the file creation date of your sitemap changes with each request etc..?

Thanks,

Sham.

 

I have been using and would recommend GSiteCrawler (find it with Google). Also sign up for Google Sitemaps. I thought my sites were pretty clean until I ran the program and found broken links and poor code that Dreamweaver did not find.

 

Yes, this is possible. You have to send the corrent xml headers that Google can recognize the format. You can even sumit sitemap.php - .htaccess rewrite rules are not necessary.

The other way around would be to create a xml sitemaps file. In your case - as I already wrote - I would write an input plugin for "kiting" release of phpSitemapNG. All you have to do is to submit the urls - with more details (e.g. lastmod, priority, ... if you like) - and phpSitemapNG will do the rest (create the sitemaps file(s) - if you have more than 50.000 pages, compress the sitemap file(s) - if you wish to, submit the sitemaps file to Google).
If you are interested in this - drop me a private message and I'll help you to set this up.

Best regards,
Tobias

 

Interesting angle for the technically adept. Rewritten URLs may pose a problem with this unless you are familiar with regular expressions and use PHP's preg_replace function (or similar). This seems like a good investment of time and energy for a select few who commonly change existing filenames.

For those unfamiliar with all of this or with filenames that remain the same over a period of time (once they the files are created) it may be a heck of a lot cheaper and easier to let one of the services like AutoMapIt handle this or to download something like GSiteCrawler and let those handle it all.

Sites running multiple software packages like CubeCart, phpBB, link directories, image galleries, etc. may have a tough time integrating several databases into this one file, but it certainly would NOT be impossible at all.

 

all my submited sitemaps are .php or .asp files..

Once your set the headers correctly it works just fine..

This way your sitemap is always up to date without having to do anything..

 

For that I would setup a seperate sitemap file for each software package and then have one sitemap index file that I submit to google..

 

I made a quick little cron job script the other day to check for new entries in my database and automatically update my sitemap. Seems to work great and I don't have to worry about anything now.

I also didn't mess with using htaccess. Instead i just used a php script and fwrite() to create a separate xml file.

 

justinewp, do you mind sharing the cron job script?

 

For what it's worth, for thos of you who use drupal, there is a gsitemap plugin at drupal.org which autogenerates your sitemap everytime google calls it.


Brian

 

Hey, I know this thread is old, but can someone help me with how to setup a phpsitemapng cron job for my site? Let me know, thanks