Help! I want a robots.txt allow only root and disallow all others

Hi there,

I have a site http://sample.com/
I want http://sample.com/ be clawed and nothing else be clawed.

So if I disallow all other, should be:
disallow: /
but if I want to allow root. then should be:
allow: /

I know it will not work. So, if you know how, please tell me. thanks

 

You can use following code:

User-agent: *
Allow: /index.php or index.html
Disallow: /

This code will block your whole website except the index page of your website that is your home page eventually.

 

thank you for your advice, the problem is no index.whatever , just root

 

i have the same problem...

also mydomain.com/index.php is not the same as mydomain.com/

 

allow only root directory and assign disallow to all the files in your site. in this way you can easily do it.