Web Games - Mortgage - iPhone Reviews - Credit Card Consolidation - Anime
PDA

View Full Version : I need a robots.txt for my proxy site. Don't know what goes in/how to do this

Browning
Dec 25th 2007, 9:34 pm
Can any of you help me out? :o
Browning
Dec 26th 2007, 9:52 am
somebody please?

I use adsense and adbrite so I was told I cannot block all bots?
lpstong
Dec 26th 2007, 10:01 am
I know you make a seperate file and call it robots txt in your 2082 cpanel -> file manager -> www. area.

Microsoft (http://support.microsoft.com/kb/217103) has simple robots tutorial.

Just be sure to only include google bot, yahoo slurp and the MSN bot. That should cover you. The rest you decide on what you want included.