1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Want to exclude TEST folder from Google indexing

Discussion in 'robots.txt' started by Melissa2007, Feb 3, 2019.

  1. #1
    I don't know much about this, which is why I'm asking here. But I want to set up a TEST folder within my domain name's main folder, for testing new website stuff before making it live.

    So for example: mydomainname folder is indexed in Google, as I want it to be, but I want mydomainname/TEST to NOT be indexed by Google. A quick and simple way of putting a robots.txt ONLY within mydomainname/TEST, to do that?

    For example, would it just say noindex, and that would prevent that folder from getting indexed by google?
    SEMrush
     
    Melissa2007, Feb 3, 2019 IP
  2. qwikad.com

    qwikad.com Illustrious Member Affiliate Manager

    Messages:
    5,683
    Likes Received:
    1,055
    Best Answers:
    20
    Trophy Points:
    400
    #2
    
    User-agent: *
    Disallow: /test/
    
    Code (markup):
     
    qwikad.com, Feb 4, 2019 IP
  3. Melissa2007

    Melissa2007 Active Member

    Messages:
    10
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    86
    #3
    Thanks quickad. Is that second line correct if I put the file inside the TEST folder only?
     
    Melissa2007, Feb 4, 2019 IP
  4. qwikad.com

    qwikad.com Illustrious Member Affiliate Manager

    Messages:
    5,683
    Likes Received:
    1,055
    Best Answers:
    20
    Trophy Points:
    400
    #4
    Yes. You're basically telling the bot to avoid indexing yoursite.com/test/
     
    qwikad.com, Feb 4, 2019 IP
  5. Melissa2007

    Melissa2007 Active Member

    Messages:
    10
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    86
    #5
    But I don't have a robots.txt in my main site, so I was putting it only in the test folder.
     
    Melissa2007, Feb 4, 2019 IP
  6. qwikad.com

    qwikad.com Illustrious Member Affiliate Manager

    Messages:
    5,683
    Likes Received:
    1,055
    Best Answers:
    20
    Trophy Points:
    400
    #6
    You need to create a robots.txt file in the main directory where your index.html (or index.php) file is. Placing that code in the robots.txt file in the TEST folder won't do any good.
     
    qwikad.com, Feb 4, 2019 IP
  7. Melissa2007

    Melissa2007 Active Member

    Messages:
    10
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    86
    #7
    Oh drat. I haven't had one in the main folder because I read that most robots just ignore them anyway. My aim with this is to keep Google from thinking that my test folder is a duplicate site, and downgrading my site.

    So if I wanted to create one for my main site, what kinds of robots do I not want? Is Google considered a robot? I don't even know.
     
    Melissa2007, Feb 4, 2019 IP
  8. qwikad.com

    qwikad.com Illustrious Member Affiliate Manager

    Messages:
    5,683
    Likes Received:
    1,055
    Best Answers:
    20
    Trophy Points:
    400
    #8
    Bots, crawlers, spiders it's all pretty much the same. Bad bots will crawl your site whether you put specifications in the robots.txt file or not. You don't have to worry about them. Google, Bing will follow the rule above.
     
    qwikad.com, Feb 4, 2019 IP
  9. Melissa2007

    Melissa2007 Active Member

    Messages:
    10
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    86
    #9
    Thanks guys! :)
     
    Melissa2007, Feb 4, 2019 IP