Robot.txt help

Hi,

I've got stuck with the robots.txt file, need some assistance.

Here is the real problem

mysite.com/example.html &
mysite.com/example/ &
mysite.com/example/anotherexample.html

I want crawlers to crawl the first url but not the other two URLs and any other file in the folder "example".

 

User-agent: *
Disallow: /example/

That will stop the lower 2 but allow the first one you listed


For more help have a read of http://www.robotstxt.org/orig.html

 

Thanks very much

 

Hello. Please show me how I rooboot.txt for this site bieberfc.com