Celebrity Pictures - Pink Ranger - Proxy - 2008 NFL Mock Drafts - Halifax

PDA

View Full Version : Don't want indexing!


webmaster
Mar 16th 2006, 2:36 am
Hi webmasters,

I dont want crawler to read some of my site's content, Because it has been used in allmost all the pages.
Is there any way to stop crawling of that text? I dont want to use frames.

Please suggest.

Thanks_

Slapyo
Mar 16th 2006, 9:36 am
You can use a robots.txt file to tell the bots what pages not to crawl. You can also place meta tags on those pages.

robots.txt
User-agent: *
Disallow: file1.html
Disallow: file2.html
Disallow: file3.html

Meta tag
<meta name="robots" content="noindex,nofollow">

webmaster
Mar 16th 2006, 11:36 am
Hi,

I want page to index but, not the content.
Is there any tactics.

Please.. suggest.

Thanks_

Slapyo
Mar 16th 2006, 12:16 pm
I think when the bot indexes the page it grabs the content. I'm not sure if it is possible to say that the bot can index the file, but not the content. Either index the file (grab the content), or don't grab it at all.

webmaster
Mar 26th 2006, 12:03 pm
Thanks alot for replying, but not get my answer.