question about google's spider(google's spider related)

Discussion in 'Google' started by so.ray2000, Nov 29, 2008.

  1. #1
    for some reason, i dont want googe's spider (and other search engines)index my site(even dont want them know my site exist),is there some ways i can use preventing google's spider away from my site?
     
    so.ray2000, Nov 29, 2008 IP
  2. tattoos

    tattoos Prominent Member

    Messages:
    1,903
    Likes Received:
    150
    Best Answers:
    0
    Trophy Points:
    335
    #2
    Open notepad, add the following code,

    
    User-Agent: *
    Disallow: / 
    
    Code (markup):
    Save as robots.txt and drop in the root of your sites directory.

    You could also add a meta tag to the head section of every page in your site like this

    <META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
    
    Code (markup):
    Hope that helps.

    Cheers
    James
     
    tattoos, Nov 29, 2008 IP
  3. so.ray2000

    so.ray2000 Peon

    Messages:
    449
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #3
    i really appreciate ur help. but how can i check whether google spiders dont index my site?
     
    so.ray2000, Nov 29, 2008 IP
  4. lycos

    lycos Well-Known Member

    Messages:
    3,769
    Likes Received:
    176
    Best Answers:
    0
    Trophy Points:
    185
    #4
    Just type the full url in the search engine. If there's no result, then there's no index.
     
    lycos, Nov 29, 2008 IP
  5. tattoos

    tattoos Prominent Member

    Messages:
    1,903
    Likes Received:
    150
    Best Answers:
    0
    Trophy Points:
    335
    #5
    Search site:yourdomain.com

    If you want your site removed from Google, you can use there url removal tool, (https://www.google.com/webmasters/tools/removals) Be careful though, it will remove it completely for a long time.. if you change your mind and want it indexed again, be prepared to wait a long time to.

    More information here http://googlewebmastercentral.blogspot.com/2008/01/remove-your-content-from-google.html

    Cheers
    James
     
    tattoos, Nov 29, 2008 IP
  6. so.ray2000

    so.ray2000 Peon

    Messages:
    449
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #6
    thank u, its really helpful. i host my multi-domains which are contained in different folders under the root, what i wanna do is keeping google index one of my domains. if i put robot.exe under the specific folder which is the domain's root, will it affect google index other of my domain(in different folder under root)?
     
    so.ray2000, Nov 29, 2008 IP
  7. tattoos

    tattoos Prominent Member

    Messages:
    1,903
    Likes Received:
    150
    Best Answers:
    0
    Trophy Points:
    335
    #7
    It should be robots.txt, not exe.

    It will not affect any of your other sites.

    Cheers
    James
     
    tattoos, Nov 29, 2008 IP
  8. so.ray2000

    so.ray2000 Peon

    Messages:
    449
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #8
    thx James...
     
    so.ray2000, Nov 29, 2008 IP