1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Need to stop some pages being indexed

Discussion in 'robots.txt' started by mogsta22, Sep 8, 2008.

  1. #1
    I have 2 pages in my site which are for prior customers only to access. They offer the product at a discount to those who have bought before.

    Both these pages are appearing in search engines (allbeit on page 32!) and this is enabling anyone to come in and get the discount right off the bat.

    How can I stop this using robots text? I expect it is something very simple and basic but I have not got a clue!

    mogsta
     
    mogsta22, Sep 8, 2008 IP
  2. imnocost

    imnocost Active Member

    Messages:
    62
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    50
    #2
    Insert the code below into the Meta Tags (at the top of your html code) in your website pages that you don't want to be indexed in the search engines.

    <meta name="robots" content="noindex,nofollow">

    Mike
     
    imnocost, Sep 13, 2008 IP
  3. amateursurgeon

    amateursurgeon Peon

    Messages:
    26
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #3
    You may need to login to google webmaster tools and request removal once you've done the above - I had problems with a page staying in the index after I had excluded it in robots.txt, this solved it.

    BTW, the way to do it with robots.txt is

    User-agent: *
    Disallow: /directory/pagename.htm
     
    amateursurgeon, Sep 15, 2008 IP
  4. himtuna

    himtuna Member

    Messages:
    225
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    28
    #4
    I tried this request as google had indexed registration form fo my site and showing it at the top results for my site name
    but they rejected saying blah blah.. this cant be done
    After some time the search results went fine.
    is there a better way to restrict the crawlers and crackers to know where is our important data ,instead of giving them robots.txt and telling them where is our safe!
     
    himtuna, Oct 6, 2008 IP
  5. Kuldeep1952

    Kuldeep1952 Active Member

    Messages:
    290
    Likes Received:
    18
    Best Answers:
    0
    Trophy Points:
    60
    #5
    If you want to prevent Googlebot and all other bots from visiting some parts
    of your web-site, you should put them in a pass-word protected directory,
    whose contents are only visible after login.
     
    Kuldeep1952, Oct 10, 2008 IP