Sitemaps - Size Important ?

[john_loch]

Hi all,

I've a question about sitemaps. I'm considering using a sitemap, pretty much for chaneling PR, but the sitemap is bound to be huge.

Anyone have observations on the limits of a sitemap - ie, an optimal number of links per page etc.

The site has a massive footprint, so I'm thinking map pages will prolly be in the vacinity of 5000 links apiece.

Perhaps it's a good idea, perhaps it's not worth the trouble.

Anyone ?

[Cricket]

Anyone have observations on the limits of a sitemap - ie, an optimal number of links per page etc.
Google is pretty specific concerning their guidelines about this.

"Offer a site map to your users with links that point to the important parts of your site. If the site map is larger than 100 or so links, you may want to break the site map into separate pages."

http://www.google.com/webmasters/guidelines.html

[disgust]

I don't know if I'd go by that 100-links thing, actually.

but definitely don't go over 100/101kb.

[Old Welsh Guy]

The google sitemap has about 140 links on it. But & I have to say this is my opinion, I would never go over the 100 link thing. Google does not say much openly, so, what it does say, it must take seriously. I understand that Google is taking more than 100k now, can't remember the exact figure though, I know Y! takes 500k.

If you have 5,000 then break it down to 50 x 100 with a link to all 50 one click from the index.

[SEbasic]

If you have 5,000 then break it down to 50 x 100 with a link to all 50 one click from the index.

That might be tricky...

two clicks would be fine...

[SEO Guy]

I would actually make it 100 x 50 heh, no seriously though Bigwebmaster and I did some tests with the link limit thing and we discovered that you could have over 800 links per page on a sitemap, still have every link count as a backlink and pass PR.

I subscribe to the no more then 100k thing, not the 100 links thing, so choose the most logical combination is it 500 x 10 100 x 50 10 x 500 it really doesnt matter, they are all spiderable and will pass the same, I just like to have the higher # in the front as it is recieving the highest PR blast so all links are much more likely to get spidered

[Old Welsh Guy]

oops, that is my bad grammar,. I meant brak it down ito 50 X100 , one click away. Meaning that the links to ALL 50 pages would be one click away for the spiders. I wouldnt actually have 100 in a page though I dont think I have gone above 70, the main being for usability to allow me to add pages within the existing links.

I have been suprised at how many people ar landing on the site maps as well, they are following bookmarked links as they are not getting there via search, rather through directly calling the url.

[webcertain]

Hi Oldwelshguy,
how could you make so many sitemaps user friendly? Wouldn't it be easier if you have a first site map with the main links and sub-site maps?
Cheers,
webcertain

[stripersonline]

how could you make so many sitemaps user friendly? Wouldn't it be easier if you have a first site map with the main links and sub-site maps?

It's sort of ironic and a testament to Google - but when you truly concern yourself with making something "user friendly", the safest thing to do is think WWGD? (what would Google do)

All of Google's rules and suggestions are designed to reward websites for being user friendly.

With 5,000 pages to put on a site map, the most user friendly thing to do would be to use good, unique titles and descriptions - then categorize and then sub-categorize and then sub-categorize - until you had groups of about 70-80 pages in each smallest unit. Why? Because your categories and sub-categories would be the natural manner for a user to find what they are looking for. It's a monumental task. But it's also the method that your users - and Google - would most appreciate :)

[peteinoz2]

when he referred to a sitemap page..

I dont think it was meant for public viewing, mainly spider crawling etc.. as a safeguard to get all his pages indexed etc..

interestingly,, I uploaded a site with 2450 pages, and as of today all pages were indexed without a sitemap.. which I was thinking of adding to the next site that goes up.. so maybe i dont need to ?

[stripersonline]

when he referred to a sitemap page..

I dont think it was meant for public viewing, mainly spider crawling etc.. as a safeguard to get all his pages indexed etc..

I'd respectfully disagree. The question asked was specifically how to make so many site maps "user friendly" - which leads me to believe they are intended to be designed for public veiwing.

[digitalpoint]

I understand that Google is taking more than 100k now....

You sure about that?

http://www.google.com/search?q=allinurl:www.digitalpoint.com/tips/filemaker.html&ie=UTF-8&oe=UTF-8

- Shawn

[wiseone]

I have the same problem... I have about 7000 pages on the site and trying to figure out how to create a good sitemap which is both user friendly and crawler friendly.

We are running into problems that our current sitemape is to deep... and the pages near the bottom (or multiple clicks from the homepage) are not getting crawled and added to the index very quickly.

[digitalpoint]

Well I would definitely say break it into pages that are less than 100k each.

[Old Welsh Guy]

Thanks for that link Shawn, I wasn't sure if Google was, thats why I said i 'thought' they might be, but the proof of the pudding there is now that they are . I read it somewhere, and can't think where.

Webcertain,
What you ask for is exactly what I suggested, Although not vry well lol. For the thrird time I will explain what I said, obviously lost in the translation.

If you have 5,000 then break it down to 50 x 100 with a link to all 50 one click from the index.

So I was saying it was better to break the list into manageable amounts of say 50 pages, and have a page that displays all of these site map pages, one click away from the index page. SO on the index page you have a link saying site map, you then hit that button and you have a nice set of links all categorised and sorted into about 50 pages. Phew , hope i explained it a little better this time.

[digitalpoint]

Thanks for that link Shawn, I wasn't sure if Google was, thats why I said i 'thought' they might be, but the proof of the pudding there is now that they are . I read it somewhere, and can't think where.

Well, what I'm saying is they *don't* spider more than 100k... It's always displayed as 101k, and if you look at the cache of it, it basically just truncates it.

[compar]

Well, what I'm saying is they *don't* spider more than 100k... It's always displayed as 101k, and if you look at the cache of it, it basically just truncates it.
I've always been fuzzy on this. Do graphic images count in the 101k total?

The old rule of thumb was that one typed page was approximately 2k. So does that mean you can have a web page the rough equilvalent of 50 typed pages -- including the html of course -- before Google will give up?

The reason I ask this is that a good site map should be all text links. It would seem to me that you could put a hell of a lot more than the 100 link limit on a 100k page. So isn't the real limit a matter of the acceptable number of links rather than the size of the page?

[digitalpoint]

Th 100k limit is just for the raw HTML page. Images, external JS files, etc. don't count toward it.

[disgust]

we have ~1,000 pages on the site.

the sitemap was over 100K with just plain text links and a small graphic for each - a very, very light page.

it's not hard to go over 100k.

[john_loch]

Hi ppl,

Nice variety of observations here. I think I'll stick to the 100K limit. Just to clarify my question a little, I should point out this is purely a PR chaneling exercise. That is to say, the sites pages have long been indexed by G|Y|Ink etc. More importantly, the navigation taxonomy is already very efficient in distributing PR where needed.

It's just that a few small segments of the site need a little boost, hence the PR chaneling. Unfortunately, the issue with large websites is the sheer volume of internal links when attempting a map.

I suppose I'm just going to have to refine my strategy :)

**** OK here's another question - If I have to get fussy, and remembering this is mainly a PR chaneling exercise, would it be wiser just to feature brief anchor text in said links (reducing file size) or should I persist with full anchor text as well ? (I mean it'd be handy - and with a site of this size, significant, but obviously expand the map tree somewhat, and impact passed PR.)

I suppose I'm verging on the PR vs Anchortext/KW advantages..

I've seen some pretty massive maps in my time, many of which completely disregard KW density issues. It's obviously a difficult affair to avoid this with a map unless the site is small, so: has anyone seen density bias in this regard from Google ? ie more flexible with site maps ?

Thanks to all who've contributed BTW :)

[compar]

Hi ppl, If I have to get fussy, and remembering this is mainly a PR chaneling exercise, would it be wiser just to feature brief anchor text in said links (reducing file size) or should I persist with full anchor text as well ? (I mean it'd be handy - and with a site of this size, significant, but obviously expand the map tree somewhat, and impact passed PR.)
Give us an example of brief anchor text versus full anchor text. In my opinion some people try to cram too much into their anchor text. I think a two, three or four word KWphrase is the absolute maximum you should use in anchor text.

For instance in my opinion many of the signature links on this forum are way too long. I don't believe that you are going to get a relevance rating for more than a few words.

[john_loch]

Give us an example of brief anchor text versus full anchor text..

OK, sure.

The links are regionally based per service.
So, I could A. Title segments of the map by service, then anchortext just the regions. Or B, I could use complete service & region anchor text per. The problem would be the repetition throughout of the service KWds. So the density would wind up through the roof for the service.

That was the ghist of it anyway :)

[compar]

OK, sure.

The links are regionally based per service.
So, I could A. Title segments of the map by service, then anchortext just the regions. Or B, I could use complete service & region anchor text per. The problem would be the repetition throughout of the service KWds. So the density would wind up through the roof for the service.

That was the ghist of it anyway :)
Well unless you combine the region and the service in each anchor text I don't think you will get found for the "regional service" search. The title of the page is not appended to the individual anchor text words or phrases. Remember this is a site map so I don't think keyword density is even an issue.

In fact I don't think keyword density is ever an issue. To suggest it is is to suggest that on-page content is a major factor in SERP placement and we know that it is infinitesimal compared to the impact of links.

[john_loch]

Thanks Compar,

I worded it badly, by title I should have said headers.. though again, the pages are already indexed, and rank for their region/service combos nicely.

After all is said and done, I think I'll go with the full anchor text, and under 100K per page should keep it below 800 links (SEO-Guy's observations earlier), and see PR propagate as needed.

Thanks everyone ! :)