1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Development site, don't want search engines...how?

Discussion in 'robots.txt' started by HiSoC8Y, Jul 12, 2013.

0
  1. #1
    Ok,

    i'm in a development stage of my site, i'll be hosting it on a VPS account.

    the domain i'll use is an old domain, where it seems it's indexed in google already. Now, that domain will be used for my testing purposes, it will be password protected, so i don't want anyone accessing it for now, not even search engines.

    how can i do that? is it through robots.txt, or is there any other method?

    have in mind, that i don't want to be blocked from google or any other search engines.

    awaiting your guidance.
     
    HiSoC8Y, Jul 12, 2013 IP
  2. robbydeletter

    robbydeletter Well-Known Member

    Messages:
    124
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    140
    #2
    just block it with robots.txt
    User-agent: *
    Disallow: /
     
    robbydeletter, Jul 12, 2013 IP
  3. HiSoC8Y

    HiSoC8Y Active Member

    Messages:
    136
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    55
    #3
    thanks robby
     
    HiSoC8Y, Jul 12, 2013 IP
  4. robbydeletter

    robbydeletter Well-Known Member

    Messages:
    124
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    140
    #4
    no prob man, we are here to help eachother. :)
     
    robbydeletter, Jul 12, 2013 IP
  5. ryan_uk

    ryan_uk Illustrious Member

    Messages:
    3,983
    Likes Received:
    1,022
    Best Answers:
    33
    Trophy Points:
    465
    #5
    One of the potential side effects of using robots.txt will be that the URLs will still be indexed, but with a notice:

    "A description for this result is not available because of this site's robots.txt – learn more."

    Although, you can use the remove URL function in Google Webmaster Tools (unless there are URL parameters involved, then you need to tweak the URL parameters section).

    Are you going to password protect with htaccess/htpasswd?
     
    Last edited: Jul 13, 2013
    ryan_uk, Jul 13, 2013 IP