Site Crawler for sites with Caching

Discussion in 'Search Engine Optimization' started by dangdatkat, Jan 20, 2010.

  1. #1
    I'm looking for a site crawler that takes into account caching query strings.

    I have been using the IIS Search Engine Optimizer (microsoft.com/web/seo/) but my content caches for users/browser and the site crawlers I have tried show duplicate pages with multiple query strings.

    Any ideas?
     
    dangdatkat, Jan 20, 2010 IP