What languages would you use to do the project below and why? Summary: Scrape a lot of data (text, pictures, videos) from multiple websites multiple times in a day (one example: all vehicle data from autotrader.co.uk) Combine all data and check for any changes in the data being scraped (e.g new pictures) Make data available on a website on a real time basis (database?) Data is searchable with multiple filters (think autotrader.co.uk again) Thank you
If you are asking about Programing Language then PHP is best to manage all your requirement for good price. ASP can be also good but hosting space or server cost a bit high for ASP.
I use Mysql for database for data of my websites. I dont think there any other better options than mysql.
Any database - MySQL, postgres, MSSql Language - PHP, python, Perl, Ruby, ASP What you're trying to do isn't a huge ask, the big issue is writing the scraper that can read the dom and extract the right info and cope with changes to the layout in a timely manner. Outputting the scraped data would be a doddle.