yahoo slurp invasion

I’m getting crawled too much by yahoo slurp:
lj511137.crawl.yahoo.net

it seems to be downloading 1/2 a gig a day, more than 700 visits per day... its killing me

I get about 20 user 10 visits form google and 700-1000 visits from yahoo, what the hell is going on!!!!!

I have tried restricting it with a robots text yesterday, but hasn’t had much of an affect today,

User-agent: Slurp
Crawl-delay: 2

How long does it take to kick in? I don’t want to ban it completely, but if its going to have a negative affect, I’m might have to ban yahoo omg

 

That does seem a little excessive. Maybe wait a day or 2 for the robots.txt to work.

 

Anyway if you don't want to limit Yahoo Slurp there is help available on-site because many people complain of this robot.

Pay a visit to Yahoo to find out more.

·​

 

Yeah just block it totally! just kidding ... you can limit it's crawl rate in your robot.txt file plenty of tutorials around on it. Good luck.

 

yup, thats what i did with this:
User-agent: Slurp
Crawl-delay: 2
Just wasn’t sure how long it would take until yahoo stopped misbehaving

 

Just wait and watch! It will take some more time for the crawling frequency to be restricted.

 

HOHO..maybe it crazy....LOL