Hi, My site have page over 100,000 & increasing day by day. I am using htaccess so there is no directory. also the Xml-sitemap generators scripts is not working showing the memory errors because the data is too much. Do you know any Idea how to make the xml sitemaps. also is there any technique so that i dont hav to make sitemap again & again rather I can append the new links to existing sitemap. Any help is really appreciated. Regards Sandeep
Ideally your sitemap is being generated from the database records of what is in your site and not by crawling the site itself. You don't have directories but your database will have a structure within it. If you are talking about the Google XML sitemap they have a restriction on the number of records that can be in one of the files so you need to check up and ensure you haven't exceeded that limit. The idea is that there is an index file and it points to all the other files that hold the actual data. Following that line of thought, you could have one of the files for 2008 pages, 2009 pages etc so that it's only the 2010 page file that needs to be rebuilt. You'd need to be very sure you had your programming right.