Hello There, This is my first posting. I just need some advice on data scraping. I would like to be able to extract names and addresses from www dot whitepages dot com dot au (seems I can't include a link yet) through the input of postcodes. I'd then like to be able to export this data into Excel to use for mailing list purposes. Can anyone tell me if this is possible, and if so, is it legal? Any advise would be much appreciated. Thanks. PS. Happy Easter from down under!
I can't find anything about the legality of using scraped addresses in snail mail in Australia, although there is something called the Spam Act of 2003 for emails. There are tons of people re-selling scraped white pages in google, and a fair amount of elance jobs, so you can probably buy the data already scraped.
Hello snowelephant, thanks for the reply. I'm new to this and didn't realise that this kind of scraped data was already available. Actually I just received an email this morning from a software company that I made some enquiries to yesterday. The software is Mozenda and he tells me that this is exactly what it is designed to do. He is going to set up a video demonstration for me . Should be interesting.
Yeah, Mozenda seems like a nice tool. I did a trial of it once for something and was happy with it. It seemed relatively easy to use compared to other scraping tools I tried, although in the end I wrote my own because of the cost, but I write software for a living, so I might have an unfair advantage
Would you be interested in selling me a version of your software? If not, how much would it cost me to get something like that coded.
The software I wrote needs a fair bit of work to hand off, and requires some programming knowledge. The white pages site provides an API, which is the same idea as scraping but the data comes out more structured. The test mode lets you run 250 queries a day, or 10k once your app is done. Here's the docs- http://developers.sensis.com.au/docs/getting_started If you want a script to extract data by postcode to excel I can write the script and get you set up for $200.
It's really not that hard if you want write your own. type in PHP Simple HTML DOM Parser Manual it has some samples you can take a look at if you want to go that route.
xintern, thanks for the reply. I did look that up, but I really don't know if I can follow it. I don't have a background in any type of programming. snowelephant, thanks for the offer. This may be the way to go for me. I have set up a 14 day trial with Mozenda and went through the tutorials, but I'm still strugling with how to use it..