Fetching huge amount of data from another website

Discussion in 'PHP' started by gr8webseller, Aug 17, 2011.

0
  1. #1
    I like to fetch lots of data from another site daily... i want to fetch text data and some images and save them to my database... that site doesn't offer official API... there is a unofficial API, but i don't like that since it is not that great... is there any other way to fetch data ?

    I heard that CURL will help... but using CURL will consume huge server resource ? do CURL can fetch image from another site and save them to my site ?

    thanks you
     
    gr8webseller, Aug 17, 2011 IP
  2. The Webby

    The Webby Peon

    Messages:
    1,852
    Likes Received:
    30
    Best Answers:
    1
    Trophy Points:
    0
    #2
    You need to understand that scrapping the content may violet copyrights of the original site. And it is illegal in most countries.
    If you understand the risk associated, and the laws that you may be braking, you can use file_get_contents() function to fetch page data.
     
    The Webby, Aug 17, 2011 IP
  3. gr8webseller

    gr8webseller Peon

    Messages:
    1,097
    Likes Received:
    7
    Best Answers:
    0
    Trophy Points:
    0
    #3
    i already aware of copyright and i am sure that i won't get such problem...
     
    gr8webseller, Aug 17, 2011 IP
  4. freelanceinphp

    freelanceinphp Member

    Messages:
    134
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    48
    #4
    cURL is only ease and efficient option to fetch data from other site.
     
    freelanceinphp, Aug 18, 2011 IP
  5. The Webby

    The Webby Peon

    Messages:
    1,852
    Likes Received:
    30
    Best Answers:
    1
    Trophy Points:
    0
    #5
    file_get_contents() is your friend then...
     
    The Webby, Aug 18, 2011 IP