We're looking for some sort of software that would allow us to scrape various pieces of content off of websites for companies that we are affiliates of. We're in the books and media space and for many companies they can deliver us a datafeed of their titles, authors, etc. However some companies can't and if we want to add their titles to our offerings we either need to do it manually (yuck...) or find some automated process. Basically what we're looking for is a tool that we could turn loose on a page and have it return certain data elements as an Excel spreadsheet. Let's use the hypothetical example of Amazon. We would be able to turn loose the bot and have scroll the Amazon site looking for product pages and then returning title, author, publisher, number of pages, date published, etc. (Note: That's just an example as Amazon already has a data feed. This is going to be used for smaller companies that don't already possess Amazon's sophistication.) One other thing: We're not interested in doing anything illegal with this software. Rather we're just looking to harvest data when/if we have the legal right to put it on our site as part of our affiliate relationship. Final thing: Ideally the software would be relatively inexpensive (under $100) and easy to use. If anyone has any any suggestions or can point me in the right direction I would greatly appreciate it!
rentacoder.com - sounds like what you need is a custom application if you want to do it right. I'd guess around $500-1000 would get it done... although thats just the tip of the iceberg, there would be a lot of extra features you could include in such an application.