1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Disable GoogleBot access without robots.txt?

Discussion in 'HTML & Website Design' started by postcd, Apr 8, 2014.

  1. #1
    Hello, i want to disable Googlebot ONLY access to certain .html page, is there any other way except robots.txt? Issue is that i dont want to publicly reveal the fact that im disallowing Googlebot access to that html page. (robots.txt is readable for anyone)
     
    postcd, Apr 8, 2014 IP
  2. COBOLdinosaur

    COBOLdinosaur Active Member

    Messages:
    515
    Likes Received:
    123
    Best Answers:
    11
    Trophy Points:
    95
    #2
    You could probably get all the IPs that the Google bots use and then block them in .htaccess; or you could check the header server side when the request comes in and return a 403 if it identifies as googlebot. Whatever you do is going to be lame and probably unreliable. I honestly cannot think of any valid reason for that kind of nonsense instead of just using robots.txt.

    When I don't want a page indexed by Google I just put the meta tag in the header:
    <meta name="googlebot" content="noindex">

    But of course anyone looking at the source will be able to see that and know your super special secret.
     
    COBOLdinosaur, Apr 8, 2014 IP
  3. Jameyson MacDonald

    Jameyson MacDonald Well-Known Member

    Messages:
    452
    Likes Received:
    83
    Best Answers:
    3
    Trophy Points:
    115
    #3
    It's an easy fix. Just go to your Google Webmaster tools and under Google Index, use Remove URLs to request that certain pages do not get indexed.
     
    Jameyson MacDonald, Apr 8, 2014 IP
    postcd likes this.
  4. postcd

    postcd Well-Known Member

    Messages:
    1,037
    Likes Received:
    9
    Best Answers:
    1
    Trophy Points:
    190
    #4
    thx, another think might be also renaming to .php and creating rule to echo the html code only if $_SERVER['HTTP_USER_AGENT'] global variable dont contain phrasse "google" (googlebot, google-mobile)
     
    postcd, Apr 8, 2014 IP
  5. blahmartinblah

    blahmartinblah Active Member

    Messages:
    431
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    93
    #5
    Good idea, but you don't need to rename it to .php. The existing file extension can be kept, and the server configured to execute .html files as php. For example:
    http://php.about.com/od/advancedphp/p/html_php.htm

    Martin
     
    blahmartinblah, Apr 9, 2014 IP