How to make search engines not crawl my site?

Discussion in 'Programming' started by andrej, Aug 5, 2011.

  1. #1
    I want to make one HTML page of my site to be not indexed by search engines. How can I do that?

    And I also wonder how to make a certain text on my site to be not indexed by search engines? What tags should I use?
     
    Solved! View solution.
    andrej, Aug 5, 2011 IP
  2. #2
    Add this line of code to your HTML:

    <META NAME='robots' CONTENT='noindex, nofollow'>
    Code (markup):
    This will stop most robots from indexing the page AND from following links on the page. If you want no indexing but following of links, then do this:

    <META NAME='robots' CONTENT='noindex, follow'>
    Code (markup):
    As far as not indexing certain parts of the page, I have no easy answer, as there are multiple ways of doing it. Simplest way is to refrain from putting non-indexable content on an indexable page.
     
    mmerlinn, Aug 8, 2011 IP
  3. wekop

    wekop Peon

    Messages:
    1,046
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #3
    wow. . a question and answer that helps at all. thanks friend was sharing here
     
    wekop, Aug 8, 2011 IP
  4. dataentryuk

    dataentryuk Greenhorn

    Messages:
    65
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    16
    #4
    Hey mmerlinn,

    Have we make a separate file for robots? or any other changes in webmaster tools?
     
    dataentryuk, Aug 8, 2011 IP
  5. mmerlinn

    mmerlinn Prominent Member

    Messages:
    3,197
    Likes Received:
    819
    Best Answers:
    7
    Trophy Points:
    320
    #5
    You can make a separate robots file, but for one page as the OP wanted, a robots page is overkill. Much simpler just to use a one-line meta tag.
     
    mmerlinn, Aug 9, 2011 IP
  6. linc

    linc Member

    Messages:
    355
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    28
    #6
    Actually there's not an efficient method to blog the indexing as long as your website has been crawled by google.
     
    linc, Aug 9, 2011 IP
  7. mmerlinn

    mmerlinn Prominent Member

    Messages:
    3,197
    Likes Received:
    819
    Best Answers:
    7
    Trophy Points:
    320
    #7
    You take the page down and replace it with a page with the correct tags at a different location. The old non-existent page will drop off google in a few months and the new one won't be indexed. You can also ask google to take down the non-existent page earlier.
     
    mmerlinn, Aug 9, 2011 IP
  8. dataentryuk

    dataentryuk Greenhorn

    Messages:
    65
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    16
    #8
    Thanks mmerlinn,

    Should we have another changes in webmaster tools.
     
    dataentryuk, Aug 9, 2011 IP