noindex, follow and did Google really understand this

hi,

I have a sitemap-page (html) which I declared "noindex, follow" within the meta-tags.

Google seems to recognize this and my sitemap is gone from the index. But I have the impression that new pages I put on the sitemap.html also do not show up in the google-index. When I remove the noindex, follow Meta-Tag new pages goes in the index more quickly.

Maybe I'm too impatiently but can please someone confirm that the syntax (noindex, follow) works with google (follow (and index of course) the links on the sitemap.html but do not index the sitemap.html itself)? It sounds funny but maybe google is offended somehow when I deceide to disallow the indexing of a page (*ggg*).

thanks for your answers and

kind regards from Vienna

Andreas.

 

Why would you not want your sitemap page indexed?

Are the pages you want to be indexed linked from the other pages on your site?

 

I want the visitors to follow either a specific path through my site or to come directly to the page they want.

So if they search for a specific keyword the SERPS should include a page from my site with the *content* they want and not a page where they can find a list of all available pages on my site.

No. Some pages should only be available through google search

Thank you

and kind regards from Vienna

Andreas.

 

You need to build links and design your site well enough so that the content pages rank higher than the site map in the search results. If your site map ranks higher than the content pages then you don't have much chance of outranking any other sites.

You can't get pages listed in google without linking to them. You need to link to all your important pages from every page on your site to make google realise they are important.

 

Yes very well understand it read what matt cutts say about it
http://www.mattcutts.com/blog/googlebot-keep-out/

 

It understand it well .I suggest that do not depend only on that rather use alternate ways alsos such as Google sitemap

 

But the last thing you want is to disallow googlebot from the sitemap

 

There is a link (from sitemap.html). A link *to* sitemap.html is on every main-menu-page.

When I put a NOINDEX, FOLLOW Meta-Tag within the sitemap.html it seems (at least for me ;-) ) that the pages that are linked from the sitemap.html (and only from them) are indexed slowly (or not at all).

When I change the Meta-Tag of sitemap.html to INDEX, FOLLOW the linked pages could be found in the google SERPS within two or three days.

So the only thing I'm changing is the INDEX/NOINDEX Tag. And my question is am I just impatient, paranoid or is there a connection?

kind regards from Vienna

Andreas.

 

Thanks for the link .. very interesting

kind regards from Vienna

Andreas.

 

Read this.

If you don't let google index the sitemap you can't expect it to crawl the links on it.

 

Instead of using a html sitemap file and not allowing Google to index it, why not use a Google Sitemap file? Or an RSS-feed that is linked from the main page?

 

Thank you all for your input. I will start a test in the next days and measure the times how long it takes to get a page into google using different meta-tags and sitemaps etc.

@softplus: I used google-sitemaps.

@mad4: In my understanding noindex does not necessarily mean that google is not allowed to *crawl* the page.

Thank you again and

kind regards from Vienna

Andreas.

 

Please have a look at my blog post at
seolion.blogspot.com/2006/02/meta-robots-tag-how-to-use-and-when_27.html

It has something on these tags

 

You are confusing noindex with no cache I think.

Sorry to say it but not allowing google to index your site map page is one of the worst seo ideas I have ever heard. You simply have to allow google to the site map page otherwise you may as well get rid of it.

 

... but there are no nofollow, noindex, follow, etc tags/attributes for Google Sitemaps (confused): https://www.google.com/webmasters/sitemaps/docs/en/protocol.html#xmlTagDefinitions

PS We ran tests on sitemaps or not last fall: http://seside.net/tests - Google Sitemaps is not the defining factor; we could not recognize any differences with Google Sitemaps or without on our test-sites with ca 900 URLs. We have seen large changes in larger sites (10k URLs+), but it is hard to set up identical test-sites with 10k URLs which are already +/- indexed evenly ... . My guess is that Google sitemaps has little or no influence on sites <1000 URLs since Google can crawl those anyway. That makes most of those online sitemap generators which are limited to 1000 URLs a bit obsolete though