I have a stand alone website on a different server which hasn't crashed from this, but for shared hosting sites this wordpress plugin will crash the site if I refresh it once after viewing it once already. and so the problem goes away once I remove the code to scrape from websites. bummer because scraping websites with this is so easy and cool. have you ever used this plugin? any way to make it less likely for it to crash ?
Forget about shared hosting companies that promise to host "unlimited websites, unlimited bandwidth and unlimited space". These are ALL LIARS! (Including the thieve company called "Hostgator"). I'm sure this scraping plugin uses too much server resources so your wordpress blog / plugin will be killed automatically. There are over 1000 websites on an average shared hosting server so if your account is using too much resources it will be banned from the server (TIP: Read your hosting company's TOS!)