Robots.txt query

[ian_ok]

Hi all,

Just a quick question here.

If I was to link to the site below *** replaces the real website, would the bot not crawl the directory I would be listed in?

Page I would be listed in....

http://www.*****.com/directory/dir-26-0.html

Robots file...
Disallow: /directory/
Disallow: /directory/links/

Thanks

Ian

[digitalpoint]

Right, it would not be crawled.

[ian_ok]

Just thought you might like to know the url from where the link is:

http://www.carslots.com

Remember to always check the robots file...

Thanks for confirming this Shawn..

Not sure if you have a Black list Shawn?

Ian

[digitalpoint]

Black list of what, sites that block spiders? I have other things to worry about (I don't do link exchanges so...) :)