In the past 2 hours or so, Ask has essentially launched a massive DDOS attack on my site. They managed to crash my server (twice) just by requesting so many pages. So, I just banned them - using robots.txt, but they can ignore that, so I added this to the top of my PHP pages: if (preg_match('#Ask Jeeves\/Teoma#', $_SERVER['HTTP_USER_AGENT'])) { die('Ask Jeeves/Teoma is evil and tends to crash servers. So, it\'s blocked. Sorry, kind of. We\'re all for competing with Google, but it\'s not our fault the spider sucks.'); } PHP: I've heard similar reports, and people complaining that Ask ignores robots.txt, and people calling them but Ask doesn't care/won't do anything.... So yeah, I'm essentially boycotting Ask until they stop sucking. Anyone care to join me?
If the AskJeeves spider requests a page from my server, it shows that error message. Therefore no CPU time or database stress will be spent on it.
I didn't research it, I wrote it. I do websites for clients and it's pretty standard PHP.... a regular expression pattern against the reported browser.