Hello everybody... I have a site that built in html. I update it very often.. I tried to make a site map using this site: http://www.xml-sitemaps.com but through the procces, it found many (alot of) pages from my forum... (if I have lets say 50 pages in my site, in the forum it found something like 400 pages a least...) What can I do? I dont want the map to inclued in itself the forum (IB forum) Is there any option? Thanks, Haim.
Im afraid I dont understand your question. If you are worried about your forum being included in the site map, don't be - generally, the more pages you have indexed, the better.
in the site that creates me the sitemap, the limit of pages can be 500 pages... so I have a problem.. and another question please.. do I need to update the site map often? or it is automattically updated somehow? thanks.. and is there a way I can make site map on my own? Thanks, Haim.
You can try A1 Sitemap Generator - not free though. However, you can schedule it through e.g. Windows Task Scheduler to run a scan and update your website sitemaps once-a-day, once-a-week or whatever. See the sitemap scan/create/upload/ping command line parameters available.
Use the paid version of the software. It is only about $15 and it is well worth it. I use xml-sitemaps on a site with over 3000 pages and it copes no problem. It also allows you to run cron jobs which I do. The only thing about cron jobs is that your host may not allow them. I run a cron job on my site every week so that the sitemap will update every week automatically but you can run it manually as well.
You can stop it parsing certain extensions (.pfd etc), you can stop it parsing any URL, you can stop it including certain extension and you can exclude certain URls and you can "Include Only" certain URL's.
Yes A1 Sitemap Generator has: Options for obeying: robots.txt "meta" tag nofollow "meta" tag noindex "a" tag nofollow Crawler filter options: File extensions String match Exact path match (optionally only subpaths) Regular expressions And it also supports "output" filters. (Used when you want to crawl e.g. entire website for links, but only wants xml sitemap generated to include parts of it.) All these options are optional of course. If you don't need special filters, just use the defaults.