Hi and hello to all of you guys in this group, I have made a software (GWebCrawler) which enables webmasters to create a Google Sitemap XML files. It's a windows application and its name is: GWebCrawler Here is the link to its website: http://www.trytobreak.com GWebCrawler & Google Sitemap Creator is a free source code web indexing engine running under the MS Windows environment. This project, is written in VB.NET. Program is currently using only one thread to browse and index the site, but in my tests, adding new threads didn't make big difference or make it any faster. Program has very high execution and running speed and it is also very small in size and very importantly it's released under the GPL Licence. As of now, this project is not very mature yet, however, it’s an one-man-project and that is why there’re still some bugs and missing features. If possible and you like this tool, please add the link to my site to yours. Thank you and have a good day everyone. Also let me know your ideas for improvement. Also, to mention. This project is written in VB.NET and I would be more than happy to share the code of this program with interested programmers in order to make this application worth the time. You can contact me at trytobreak @ gmail.com or email address of this post. Sincerely, Joe Jarosciak
Looks good. Does this tool also look to see if the urls are greater than 50,000? This is one of the limitations large sites have. It'd be nice to include support for breaking up the site maps into separate .xml files once 50,000 urls have been reached.
It doesn't have a limit on number of pages and support for cutting of sitemaps to 50k url files shouldn't be hard to do. Currently it would be the best if someone wanted to help on this project. I can provide source code in VB.NET. Joe
you have to put every single folder to crawl all the site? that's a bit amateur or i didn't understand how it works