robot.txt

[sportsguru]

I hope this is the right forum for this question. Is there some way using a robot.txt file to exclude a directory from finding my index page? Reason being I have asked this certain directory to please remove my listing and for some weird reason they won't do it?!? They say they can list whatever site they want, etc.

I've heard you can prevent certain engines or pages from crawling your pages; but what about a directory? Can this be done? If so, I know next to nothing about robot.txt. Can someone explain how to set this up?

[Crazy_Rob]

robots.txt is just directions for robots. It's not actually going to keep anyone out.

You can use .htaccess to ban certian IP's or user_agents. But that's not going to help you in this case.

[redking]

I hope this is the right forum for this question. Is there some way using a robot.txt file to exclude a directory from finding my index page? Reason being I have asked this certain directory to please remove my listing and for some weird reason they won't do it?!? They say they can list whatever site they want, etc.

I've heard you can prevent certain engines or pages from crawling your pages; but what about a directory? Can this be done? If so, I know next to nothing about robot.txt. Can someone explain how to set this up?

If they won't change a listing you might have to take legal action.

[fryman]

Legal action? Why? What are they doing wrong?

[riziko]

Can .htaccess block by referrer? That might work.