Blocking the search engines from indexing the websites

Discussion in 'Search Engine Optimization' started by spotlight, Oct 20, 2009.

0
  1. #1
    How we can block the all search engines from being indexed websites

    I would like to know in detail
     
    spotlight, Oct 20, 2009 IP
  2. TheCrabb

    TheCrabb Peon

    Messages:
    304
    Likes Received:
    13
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Robot.txt - NoIndex, NoFollow metatags
     
    TheCrabb, Oct 20, 2009 IP
  3. affiliates4seo

    affiliates4seo Peon

    Messages:
    248
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Using of robots.txt is the best way.

    Write:

    User-agent: *
    Disallow: /

    It restricts your entire site.
     
    affiliates4seo, Oct 20, 2009 IP
  4. sherone

    sherone Well-Known Member

    Messages:
    1,539
    Likes Received:
    16
    Best Answers:
    0
    Trophy Points:
    130
    #4
    disallow in robots.txt file. Thats the best way.
     
    sherone, Oct 20, 2009 IP
  5. webdev007

    webdev007 Active Member

    Messages:
    1,037
    Likes Received:
    13
    Best Answers:
    3
    Trophy Points:
    88
    #5
    Disallow in robots.txt file but also be sure that there should be no sitemap file submitted to SEs.
     
    webdev007, Oct 20, 2009 IP
  6. affiliates4seo

    affiliates4seo Peon

    Messages:
    248
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #6
    Even, if submit a sitemap also search engines doesn't crawl the site. But, it gives few crawl access erros in sitemap at Google webmaster tools.
     
    affiliates4seo, Oct 21, 2009 IP
  7. seojohnny2009

    seojohnny2009 Peon

    Messages:
    228
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #7
    yes robot.txt can do this for u
     
    seojohnny2009, Oct 21, 2009 IP
  8. Jeff Collision

    Jeff Collision Peon

    Messages:
    1,020
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #8
    I found this site and might be useful for you Robots.txt....
     
    Jeff Collision, Oct 21, 2009 IP