How to prevent SEs from indexing parts of your site?

Discussion in 'Search Engine Optimization' started by jasonwilks, Jan 4, 2007.

0
  1. #1
    How can I prevent search engines from indexing some of the pages in my site? Is there a meta tag I need or something? Please help!
     
    jasonwilks, Jan 4, 2007 IP
  2. Austars

    Austars Active Member

    Messages:
    1,437
    Likes Received:
    23
    Best Answers:
    0
    Trophy Points:
    95
    #2
    Austars, Jan 4, 2007 IP
  3. jasonwilks

    jasonwilks Well-Known Member

    Messages:
    65
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    111
    #3
    hmm, thanks... I just read that this meta code:

    <META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">


    a robot should neither index this document, nor analyse it for links.

    So If I put this code in the pages that I dont want google (or any other SE) to index, then it wont index them correct? Or do I also need another file/program to go along with this?
     
    jasonwilks, Jan 4, 2007 IP
  4. Alberta

    Alberta Peon

    Messages:
    92
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Hi There!

    You could create the 'no follow' within each page (more work)...but as was mentioned by creating a robots text file, you only need to make changes to one file.

    A
     
    Alberta, Jan 4, 2007 IP
  5. jasonwilks

    jasonwilks Well-Known Member

    Messages:
    65
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    111
    #5
    Ok thanks I have everything all figured out, I got a robots.txt file along with that meta code now.
     
    jasonwilks, Jan 4, 2007 IP
  6. roy77

    roy77 Active Member

    Messages:
    1,088
    Likes Received:
    15
    Best Answers:
    0
    Trophy Points:
    78
    #6
    theres something called robots.txt that tells search engines which part to crawl and which one not to.
     
    roy77, Jan 4, 2007 IP
  7. dcristo

    dcristo Illustrious Member

    Messages:
    19,776
    Likes Received:
    1,200
    Best Answers:
    7
    Trophy Points:
    470
    Articles:
    7
    #7
    I would use robots.txt to disallow bots from crawling certain folders.
     
    dcristo, Jan 4, 2007 IP
  8. northbeach23

    northbeach23 Banned

    Messages:
    135
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #8
    Thats right! Use robots.txt for the page you dont want to crawl.
     
    northbeach23, Jan 5, 2007 IP