Loans - Loans - Car Accident Lawyer Los Angeles - Loans - Cell Phones

PDA

View Full Version : Excluding subdmains


frio
Oct 30th 2007, 2:40 am
how would i be able to exclude the subdomain from crawling?
can I just put this on the robots.txt file:

disallow: subdomain.domain.com

I think it wont work. can you suggest any other things about this?

Thanks

leo
Oct 30th 2007, 2:43 am
What's letting you think it wouldn't work?

frio
Oct 30th 2007, 9:55 pm
What's letting you think it wouldn't work?
would it work? what you think?

Kuldeep1952
Oct 30th 2007, 11:26 pm
To disallow a Sub-domain from crawling, you have to put the robots.txt file
in the root of the sub-domain.

To disallow all files, add the follwoing:

User-agent: *

Disallow: /