Hello, I have this site (mercedesonsale.com) and Google only indexed my index.html page. I have a lot of other pages on the site that weren't indexed. Any clue on why this appened? I have it om Google sitemaps and all that stuff. Thank you, Ralph Nunes
Depends on how long it's been up. Also you can try getting links to those pages you want indexed. There might also be a problem with your php code. Thank You
I had a look at your site www.mercedesonsale.com, and here's what I found: Your robots.txt file is not accessible (it redirects to the index page instead). Search engines won't like this. Have you check the site diagnostics within Google Sitemaps? Fix the errors in your HTML code, as found by the W3 Validator. Again, search engines really don't like pages that don't parse correctly. They don't really like nested tables either, so change that if you can. How long has your site been up? It can take a while before they index all the pages - links from this forum and other sites can sometimes speed things up. Cheers, Cryo.
Could you give an example of a robots.txt file to index all the files, please? And is there a software to automatically correct the HTML source?
You do not need a robots.txt if you want all your pages to be indexed. My advise is to create a google sitemap and check the last visit of the bots.
First of all you should say when your site was started If it was started before month no need to worry as Google takes a lot of time to index the inner pages but if was started before 3 month then you should look into that Prepare a sitemap and submit to Google and also try posting your inner page url in any forum Then I am sure Google will index your website
I've had web clients complete dissapear from Google due to not having a blank robots.txt file, which ended up being a custom 404 that wasnt done right. Something in the custom 404 told spiders to not index anything.