Google had indexed 3 versions of my site while I was building it: xttp://MYSITE.com xttps://MYSITE.com xttp://www.MYSITE.com Since then I have: 1. used 301-redirects to establish xttp://www.MYSITE.com 2. robot.txt restricted individual pages that are xttps:// to prevent it from making the switch to secure mode. 3. Put on google that I want my site to be listed as http://www.MYSITE.com 4. Made a Google Sitemap to link only to the pages I want indexed. My questions: 1. Am I getting penalized for duplicate content right now? 2. Will the wrong versions eventually go away? 3. It Robots.txt enough to stop the xttps:// part of the sites not to be crawled or is the noindex meta necessary? I know nobody knows how the googlemonster works but I'm sure someone has gone through this before?
1. without given URL no one can tell anything for sure, you may be penalized, or you may not 2.The wrong versions will be removed after some time. If the error lasted one, or two days, the situation would be normal in say one week. If you had it for more than a week, the situation is more grave. 3.For your problem the sitemap is irrelevant. Robots.txt would be enough, but as the error has already been made meta tag is necessary, as the bots can pass by robots.txt file. [spiders do not ignore robots.txt, but if they do not see it, they index everything.] So far, if you see indexed pages in Google it is OK. I hope that after 301 redirection everything will be fine.
Thanks a ton for the comment keym@k3r and +rep for ya Sadly, I've had some URLs and the https/non-www versions listed in Google for about 2 weeks now despite my telling Google not to by the above methods. My URL is xttp://www.persianamania.com -
Well, thank you. The site is indexed with site: and info: results. So my guess it is not punished, but it will take some time to reindex the pages. BTW the tools for server checking you can find on oyoy.eu.