1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

How to block subdomain in Robots.txt

Discussion in 'Search Engine Optimization' started by PromoDirect, Nov 2, 2011.

  1. #1
    User-agent: *
    Disallow: /

    =================

    This is how my current robots.txt file looks like. I want to block a subdomain (i.e http://subdomain.mywebsite.com) using robots.txt, I don't want even a single page of my subdomain to be indexed in Google.

    I added the above text to prevent my Subdomain from indexing... but it still got indexed into Google. What should I do to entirely prevent my Subdomain from Indexing.
     
    PromoDirect, Nov 2, 2011 IP
  2. website-design-seo

    website-design-seo Peon

    Messages:
    131
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Hello..Please You should Do this..
    User-agent: *
    Disallow: /your subdomain name
     
    website-design-seo, Nov 2, 2011 IP
  3. katich.sm

    katich.sm Greenhorn

    Messages:
    55
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    18
    #3
    Hi PromoDirect,

    you can use same robots.txt file it will help to black subdomain

     
    katich.sm, Nov 2, 2011 IP
  4. waqas0323

    waqas0323 Peon

    Messages:
    47
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Create a second robots.txt for the subdomain.
    robots.txt can only contain instructions for the folders/sub folders,
    not sub domains.
     
    waqas0323, Nov 2, 2011 IP
  5. webdev007

    webdev007 Active Member

    Messages:
    1,037
    Likes Received:
    13
    Best Answers:
    3
    Trophy Points:
    88
    #5
    Subdomains are treated as a different sites. You can use the same code (Disallow: / ) but make sure it works with "subdomain . mywebsite . com/robots.txt" only and not with "www . mywebsite . com/robots.txt" etc., else you main site pages will also gets blocked.
     
    webdev007, Nov 2, 2011 IP
  6. Redberry-small

    Redberry-small Greenhorn

    Messages:
    57
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    16
    #6
    After creating a new robots.txt file for your subdomain do not forget to ust "Delete URL" option in your Google Webmaster as disallowing pages from indexing in robots does not delete those pages which have already been indexed and cached by Google.
    Good luck
     
    Redberry-small, Nov 2, 2011 IP
  7. PromoDirect

    PromoDirect Member

    Messages:
    267
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    28
    #7
    Do you mean
    Instead of:
    User-agent: *
    Disallow: /


    I must have
    User-agent: *
    Disallow: /http://subdomain.mywebsite.com
     
    PromoDirect, Nov 3, 2011 IP
  8. Richard Matt

    Richard Matt Peon

    Messages:
    21
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #8
    Yes I have the robots.txt file uploaded in the root of my Sub Domain and not the main domain.
    I have done this for more than one month now and still have 23,600 pages indexed in Google. 23,600 has been showing up since the time I uploaded the robots.txt file, the number never seems to be going down and thats what making me worry.
     
    Richard Matt, Nov 3, 2011 IP
  9. PromoDirect

    PromoDirect Member

    Messages:
    267
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    28
    #9
    Could you let me know how does your robots.txt looks like? I mean what condition have you stated in your robots to block the subdomain? Like I have stated in my last post above.
     
    PromoDirect, Nov 9, 2011 IP