I submitted a new sitemap 2 days ago, and also set up global redirects on all old urls. Any thoughts on why this is the case? Also, does anyone out there know of a free crawling program that could replicate a spider and give me more insight?
A few things to check. First, make sure that all pages in your sitemap have a unique title. Pages with identical meta titles may not always be indexed. Second, make sure that each page of your site can only be accessed through 1 URL. For example, they should be able to get to the same product using the following 2 URLs... www.example.com/product_one www.example.com/most_popular/product_one These pages may be indexed only once, and sometimes not at all. Third, make sure each of the pages in yoru sitemap can be access in a few seconds. If it takes longer than a few second to access, the pages may not be accessed. Fourth - Check your robots.txt and no index pages and make sure your sitemap doesn't include pages that are restricted by robots.txt or noindex. Fifth - make sure pages included in the index can be accessed by someone not logged in. Hope this helps.