1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

What is the best language for this huge requirement? Your expert input required please

Discussion in 'Programming' started by WHoSaidThis, May 21, 2020.

0
  1. #1
    What languages would you use to do the project below and why?

    Summary:
    • Scrape a lot of data (text, pictures, videos) from multiple websites multiple times in a day (one example: all vehicle data from autotrader.co.uk)
    • Combine all data and check for any changes in the data being scraped (e.g new pictures)
    • Make data available on a website on a real time basis (database?)
    • Data is searchable with multiple filters (think autotrader.co.uk again)
    Thank you
     
    WHoSaidThis, May 21, 2020 IP
  2. seomanualsubmission

    seomanualsubmission Well-Known Member

    Messages:
    904
    Likes Received:
    128
    Best Answers:
    4
    Trophy Points:
    165
    #2
    If you are asking about Programing Language then
    PHP is best to manage all your requirement for good price. ASP can be also good but hosting space or server cost a bit high for ASP.
     
    seomanualsubmission, May 21, 2020 IP
  3. WHoSaidThis

    WHoSaidThis Peon

    Messages:
    2
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    1
    #3
    Thanks. PHP because it’s fast ?

    what would you use for storage of the data?
     
    WHoSaidThis, May 22, 2020 IP
  4. seomanualsubmission

    seomanualsubmission Well-Known Member

    Messages:
    904
    Likes Received:
    128
    Best Answers:
    4
    Trophy Points:
    165
    #4
    I use Mysql for database for data of my websites. I dont think there any other better options than mysql.
     
    seomanualsubmission, May 22, 2020 IP
  5. sarahk

    sarahk iTamer Staff

    Messages:
    28,500
    Likes Received:
    4,460
    Best Answers:
    123
    Trophy Points:
    665
    #5
    Any database - MySQL, postgres, MSSql
    Language - PHP, python, Perl, Ruby, ASP

    What you're trying to do isn't a huge ask, the big issue is writing the scraper that can read the dom and extract the right info and cope with changes to the layout in a timely manner.

    Outputting the scraped data would be a doddle.
     
    sarahk, May 23, 2020 IP