I'm trying to remove a site from the search engines index. Putting in a disallow all command in the robots.txt worked for Google. But it's still there in Microsoft and Yahoo. Will it eventually disappear in Yahoo or is there something else I need to do?
You should adapt your robots.txt file appropriately - see www.robotstxt.org/wc/norobots.html for example. Quite simple task indeed, you will see. The other way round is much more laborious!-)
You beat me here . Well but, why would you not want the spiders to know about your site .. Ok .. ok got it.. privacy . ~G
Nope, I want it to make way for another site of mine. Then I'm closing it down. I don't think robots.txt will work in Yahoo for this purpose. It seems you have to first authenticate then delete it in their Site Explorer tool.
Hey! It's really interesting the robots.txt thing. Must the file be in the same directory as the index?