I do not want my site on search engines - how can i do this

Discussion in 'Search Engine Optimization' started by iloveunicorns, Sep 29, 2009.

  1. #1
    I plan on creating a personal site. I do not want the site the site one any search engine. :confused:

    how do i accomplish this ... or is this impossible

    thanks
     
    iloveunicorns, Sep 29, 2009 IP
  2. Visagoo

    Visagoo Peon

    Messages:
    51
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Visagoo, Sep 29, 2009 IP
  3. iloveunicorns

    iloveunicorns Peon

    Messages:
    270
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #3
    so that will disallow all search engines right
     
    iloveunicorns, Sep 29, 2009 IP
  4. alexts

    alexts Well-Known Member

    Messages:
    1,126
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    125
    Digital Goods:
    1
    #4
    it will disallow all search engine spiders who are following robots.txt file (most legit search engines do)
     
    alexts, Sep 29, 2009 IP
  5. webwest

    webwest Peon

    Messages:
    55
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #5
    You can also put this in the head of your page(s):

    <meta name="robots" content="noindex, nofollow" />
     
    webwest, Sep 29, 2009 IP
  6. Rudhra

    Rudhra Peon

    Messages:
    1
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #6
    use google webmaster tool to not index your site
     
    Rudhra, Sep 29, 2009 IP
  7. gordon08

    gordon08 Peon

    Messages:
    529
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    0
    #7
    Password-protect your pages using htaccess.
     
    gordon08, Sep 29, 2009 IP
  8. snowbird

    snowbird Notable Member

    Messages:
    3,036
    Likes Received:
    395
    Best Answers:
    0
    Trophy Points:
    290
    #8
    Best to use the noindex meta tag on all your pages. Google may not obey the robots.txt file and include your pages (without descriptions) in their search engine.
     
    snowbird, Sep 29, 2009 IP
  9. extomas

    extomas Member

    Messages:
    237
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    33
    #9
    The best way of doing it is disallow entire website through robots.txt.
     
    extomas, Sep 29, 2009 IP
  10. Canonical

    Canonical Well-Known Member

    Messages:
    2,223
    Likes Received:
    141
    Best Answers:
    0
    Trophy Points:
    110
    #10
    Oh yea.. that's a great approach! NOT.

    Unfortunately, this solution won't keep Yahoo!, Bing, and others from indexing the site.



    The best ways to prevent this from happening is:

    1) Use the robots.txt that a previous poster suggested:

    (in which there can be cases where your URL still shows up in the SERPs)

    2) Add <meta name="robots" content="noindex"> elements to ALL pages on your site (pain in arse since have to remember to do so every time you add a page)

    or

    3) Password protect your site (either by adding a login screen or setting a password at the folder level on your server) and ONLY give the password to those you want to have access. Crawlers are NOT going to be able to login.

    The first two approaches will likely be ignored by any "bad bots" and can still result in your pages getting indexed in by some obsure, bad crawlers.

    The last approach will insure ALL crawlers (good AND bad) are unable to crawl and index your pages.
     
    Canonical, Sep 29, 2009 IP
  11. iloveunicorns

    iloveunicorns Peon

    Messages:
    270
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #11
    okay thank you guys

    i will use the robot.txt method

    thanks canonical

    will follow your three methods
     
    iloveunicorns, Sep 29, 2009 IP
  12. ErikJ

    ErikJ Peon

    Messages:
    1,539
    Likes Received:
    14
    Best Answers:
    0
    Trophy Points:
    0
    #12
    also there are many people who try to get traffic from search engines and do not. The above is good advice but also if you do nothing the chances of your site getting into all the search engines is very low
     
    ErikJ, Sep 29, 2009 IP