part of my site refuses to be indexed

i can see both sides of this, but still would have thought that getting pages indexed that weren't otherwise could only help, even if you then go about sorting the structure out as per...?

surely if you have pages not indexed, your site structure is even more broken as far as G is concerned due to "broken" links pointing at those pages, and the links on the missing pages not letting any juice flow through to anywhere?

and then again once you have fixed it to the best you can, why not install one then to ensure as little future droppage as possible?

its not something have heard anyone say before

 

I own and have optimized many sites in excess of 1 Million pages, and sitemaps really hamper the task. As i said, if a webpage is only in the index because it got found via an XML sitemap is that going to be a strong and well performing page? No of course not, and chances are it will rank for nothing so what good is it by being simply indexed?

You want authority to be passed to these pages so they actually rank and pull traffic, and the only way to do this is with links plain and simple.

The best way to identify these pages or sections of your site is they are not indexed, or take a long time to index after creation. Then you can create keyword anchored links in to this section from your stronger pages which creates a flat file structure. Note Wikipedia is a good example of this, and speaking of Wiki can you find their XML sitemap?

I strongly urge you to read this thread: 4,185 Page #1 Rankings in 14 Days

This was done primarily using internal linking with data that would of been lost had an XML sitemap masked the issues by indexing problematic sections. This site is now running on 4 clustered webservers, they were on 1 before.

Exactly, and an XML sitemap will not fix the problem or make it go away. At least with them "not" indexed, you know exactly where to drive links to in order to create an overall authoritative domain.

It boils down to what would you rather, a weak page that ranks for nothing but is indexed just because of a sitemap or a page you can instantly identify as problematic and know to drive some targeted anchor text links to?

 

of course. makes good sense. and one mill pages, wow

never been on anything like that scale.

 

sweetfunny, your write up on long tail marketing is excellent, something every SEO needs to know.

I believe I understand where you're coming from now. You're saying not to use XML sitemaps, because it will mask those pages which are not being indexed, or are targetted improperly, or are simply underperforming. Now I get what you're saying, and I do have to agree. A sitemap is no solution for those problems.

Quite frankly, I've never worked on sites which had the problem of not being linked or indexed properly, because I always make sure they do; I only used sitemaps as an additional tool, since, as Google states "they can never hurt you." I can see why, in your case of reviewing indexed results for performance, sitemaps actually can be a hindrance.

Thanks, and bravo!

 

Thanks, and yes it does go against what most people say about sitemaps but data is a SEO's best friend.. You can "never" have too much of it, and if pages of a site or complete subsections are having indexing problems i want to know about it and fix it instead of simply getting it indexed via the back door.

Typically a site that's internally linked optimally is also human friendly as well, because content is easily accessible and it's simple to navigate from A to B which is an added benefit besides SERP's and PR.

Wikipedia and W3C are good examples, they have dozens of words in each document hyperlinked with keyword rich anchor text to other related pages. Following this is a great strategy, if pages or sections aren't performing well edit some of your stronger pages and turn words in to anchor rich backlinks to the lesser pages.

HTML Sitemaps are excellent, especially included as 404 error pages because they do basically everything an XML sitemap does plus they are human friendly, they pass Pagerank, they provide anchor text backlinks, they create a flat file structure and so on.

I'm glad you can see where i'm coming from with the XML's.

I actually know a SEO in the Netherlands, and the only time he submits XML sitemaps is when he wants pages "removed" from the SERP's faster. He submits the URL's that no longer exist so Google gets the hint to remove them. And you thought my methods were backwards.