1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

How to create robot.txt file ? how to stop indexing files from cgi-bin ?

Discussion in 'robots.txt' started by poseidon, Jan 23, 2006.

0
Page 2 of 2
  1. MrPJH

    MrPJH Well-Known Member

    Messages:
    1,066
    Likes Received:
    7
    Best Answers:
    1
    Trophy Points:
    155
    #21
    i cam here to ask the same question but found it already and most helpful
    can i disallow: filename.php too? or only directories have to disallow
     
    MrPJH, Apr 10, 2009 IP
  2. linkdealer

    linkdealer Active Member

    Messages:
    138
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    90
    #22
    You can disallow the pages as well along with directories
     
    linkdealer, Jun 9, 2009 IP
  3. kiwin

    kiwin Peon

    Messages:
    281
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #23
    hello, im new to this stuff too....you can check bigger sites like: http://www.cnn.com/robots.txt for more info.
    and what is the difference between "/cgi-bin/" and "/cgi-bin" which is the right one?
     
    kiwin, Jun 14, 2009 IP
  4. sgtcory

    sgtcory Peon

    Messages:
    9
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #24
    That depends on how it's being used. For example - if you have a Perl script you want to call, you couldn't call it without typing the full directory i.e. /cgi-bin/. This is because the server needs to know that it is a directory to look in for the xxxxxx.pl / xxxxx.cgi you want to run. Otherwise your call would look like :

    /cgi-binxxxxx.pl

    When you really want :

    /cgi-bin/xxxxx.pl

    However - leaving the trailing slash off is sometimes required if you are running a perl script in an server environment that already appends the slash (which is not typically the case). Or for example - you have a call to a perl script from a php script that looks like this :

    $script_call='/xxxxx.pl';
    if {
    $thishappens;
    }
    echo '/cgi-bin$script_call';

    Make sense?
     
    sgtcory, Jun 14, 2009 IP
Page 2 of 2