I need a robots.txt for my proxy site. Don't know what goes in/how to do this

Discussion in 'robots.txt' started by Browning, Dec 25, 2007.

0
  1. #1
    Can any of you help me out? :eek:
     
    Browning, Dec 25, 2007 IP
  2. Browning

    Browning Banned

    Messages:
    1,928
    Likes Received:
    29
    Best Answers:
    0
    Trophy Points:
    0
    #2
    somebody please?

    I use adsense and adbrite so I was told I cannot block all bots?
     
    Browning, Dec 26, 2007 IP
  3. lpstong

    lpstong Notable Member

    Messages:
    3,292
    Likes Received:
    216
    Best Answers:
    0
    Trophy Points:
    230
    #3
    I know you make a seperate file and call it robots txt in your 2082 cpanel -> file manager -> www. area.

    Microsoft has simple robots tutorial.

    Just be sure to only include google bot, yahoo slurp and the MSN bot. That should cover you. The rest you decide on what you want included.
     
    lpstong, Dec 26, 2007 IP