avoid robots to index mainsite.com/undersite.com

Discussion in 'robots.txt' started by dalailama, Jan 11, 2008.

0
  1. #1
    Hi All!

    I have set up a new site at my webhost. The problem is that the site is accessabel from the adress mainsite.com/newsite.com

    I want prevent that adress and all subsites on the new site to be indexed by robots, just to have them index the new site´s domain name.

    How can I fix this with robots.txt?

    Thanks in advance! :)
     
    dalailama, Jan 11, 2008 IP
  2. Pwner

    Pwner Peon

    Messages:
    176
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    0
    #2
    You can try with something like...
    User-agent: *
    Disallow: /newsite.com/
     
    Pwner, Jan 11, 2008 IP
  3. SwapsRulez

    SwapsRulez Peon

    Messages:
    32
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #3
    +1... also you can write as many subsites you want....

    like the below example..

    User-agent: * 
Disallow: /newsite.com/
Disallow: /newsite1.com/
Disallow: /newsite2.com/
Disallow: /newsite3.com/
    Code (markup):
    Peace!
     
    SwapsRulez, Jan 12, 2008 IP
  4. rich_uk

    rich_uk Peon

    Messages:
    321
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Just tro throw this into the mix, if you DIDN't want the waybackwhen machine indexing your site just add
    
User-agent: ia_archiver
Disallow: /
    Code (markup):
     
    rich_uk, Jan 23, 2008 IP