Robots query...

I've been advised to use the following in my robots.txt file

User-agent: *
Disallow: /test/
Disallow: ./test/

I believe the 1st disallow means BOTS do not visit anything in the folder TEST but do VISIT the sub folders

Does the 2nd one mean BOTS do not visit anything in TEST and any other sub folder.

I've read the robots.org page but with subfolders I'm lost.

So lets say in a folder called SHOP I have 3 .php files and I want to disallow to one I would do this

Disallow: /SHOP/1.php

That way 2.php and 3.php would be indexed

Also within a webpage does the command NO FOLLOW mean the page will no longer be indexed, but it will follow and links within it?

Thanks.

Ian

 

The first one there will disallow that directory and everything in it including sub directories. The second one I'm not sure. I haven't seen or used that method before.

That's correct.

No, the opposite is true. The page will be indexed, but the links will not be followed.

 

Disallow cannot contain relative paths (e.g. ./test or test). It may only contain partial absolute paths (e.g. /test will match /test.html and /test/test.html, while /test/ will only match /test/test.html, but not /test.html).

Search this forum for robots.txt and you will find numerous discussions and examples about robots.txt.

J.D.