Robots.txt advice

Discussion in 'Search Engine Optimization' started by ianv, Mar 28, 2011.

0
  1. #1
    Hi

    How would I stop a robots.txt from indexing a test site on my domain as its causing duplicate content issues.

    For example how do you stop google from indexing

    http://testsite.mydomain.co.uk in the robots.txt file - I have not got a robots.txt file at the moment
     
    ianv, Mar 28, 2011 IP
  2. DocuMaker

    DocuMaker Active Member

    Messages:
    427
    Likes Received:
    16
    Best Answers:
    0
    Trophy Points:
    60
    #2
    Place the following in a robots.txt file and then upload it to your server root:

    User-agent: *
    Disallow: /
     
    DocuMaker, Mar 28, 2011 IP
  3. Excel 8

    Excel 8 Guest

    Messages:
    1,231
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    0
    #3
    it happen all the time even though you are not doing SEO to that...

    thanks!
     
    Excel 8, Mar 28, 2011 IP