Robots txt for Sub-domain

Discussion in 'Search Engine Optimization' started by vickyseo, Sep 3, 2008.

  1. #1
    Hi all,
    I have two sub-domains in my main domain, I want to disallow my both sub-domains from SE crawler or spider but I want to allow for main domain so what action I do…

    Please suggest me…
     
    vickyseo, Sep 3, 2008 IP
  2. DrAnujGupta

    DrAnujGupta Peon

    Messages:
    203
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #2
    you are expert showing in signature and looking for robot.txt file.
    Robot.txt can help you. sure.
     
    DrAnujGupta, Sep 3, 2008 IP
  3. justinlorder

    justinlorder Peon

    Messages:
    4,160
    Likes Received:
    61
    Best Answers:
    0
    Trophy Points:
    0
    #3
    robotstxt.org for reference.
    google.com/robots.txt
     
    justinlorder, Sep 3, 2008 IP
  4. Kneoteric_eSolutions

    Kneoteric_eSolutions Banned

    Messages:
    382
    Likes Received:
    10
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Good point there Anuj. I think Vicky has his answer now.
     
    Kneoteric_eSolutions, Sep 3, 2008 IP
  5. Australianfranchises

    Australianfranchises Peon

    Messages:
    1,230
    Likes Received:
    7
    Best Answers:
    0
    Trophy Points:
    0
    #5
    yes it will help robots.txt file.you can try!
     
    Australianfranchises, Sep 3, 2008 IP
  6. Bloomtools

    Bloomtools Peon

    Messages:
    1,361
    Likes Received:
    12
    Best Answers:
    0
    Trophy Points:
    0
    #6
    Sub domain has to be submitted individually from the main domain.
    That’s why you also have to submit separate robots file!
     
    Bloomtools, Sep 4, 2008 IP
  7. www.rapidvectorseo.com

    www.rapidvectorseo.com Peon

    Messages:
    312
    Likes Received:
    7
    Best Answers:
    0
    Trophy Points:
    0
    #7
    Subdomain root directory robots.txt should include,

    User-agent: *
    Disallow: /

    and


    Main root directory robots.txt should include:

    User-agent: *
    Disallow: /subdomain_directory/
     
    www.rapidvectorseo.com, Sep 4, 2008 IP
  8. genuineseo

    genuineseo Well-Known Member

    Messages:
    783
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    108
    #8
    yah i agree with rapidvectorseo! you can follow that one!
     
    genuineseo, Sep 4, 2008 IP