I am working on several sites and I am trying out SOFTplus GSiteCrawler at the recommendation of someone in a previous post and now comes a new question from me.. How does Google handle directory structure? I have several sites that seem to use the same format: URLs with duplicate content (identical pages): http://www.sitename.com/directory/ http://www.sitename.com/directory/index.asp The program tells me those pages have duplicate content, however technically they are the same page. I would think Google would know that, however maybe it doesn't? Should I stick with my current format or should i rename every single link that isn't fully named? With that in mind, if my site is http://www.sitename.com/index.asp for the main page, but you can type in http://www.sitename.com to get to that page, how does that work out. Grr Google, you confuse me
I personally would use 301 redirect from http://www.sitename.com/index.asp to http://www.sitename.com so that all the link love goes to http://www.sitename.com. Some people might have linked to the index.asp version and this solution makes sure everything gets bundled up in one page. As with your internal links, I wouldn't make it because of the duplicate content but rather so that your pages rank better. Otherwise you have some links pointing to one version and some pointing to the other.