I don't want the bots to index any pages with "tetrahost.net/" on it but I want them to follow all of the gamers.puck3d.net links, ok. I have gamers.puck3d.net as an addon domain for the entire site, meaning this page: http://gamers.puck3d.net/agora/ and gamers.tetrahost.net/agora/ will look exactly alike, thus creating duplicate content..... which I don't want. would this code successfully do that goal? User-agent: * Disallow: http://gamers.tetrahost.net/ Code (markup):
You need to have two seperate robots.txt on each site, one which blocks robots, one that doesn't. The tricky thing is that these two files need to have the same filename and live in the same directory. You can probably use mod_rewrite to do this, but serving either robots1.txt or robots2.txt depending on which host has been requested. Check the FAQ for how to do this. Cheers, Cryo.