Screen Scraping with CURL

Discussion in 'PHP' started by elambert, Feb 3, 2006.

  1. #1
    Hi,

    I am new to PHP and I using the CURL library for screen scraping.

    Specifically, I want to get a webpage, pull some information off of it, and then display the page.

    Unfortunately, when I display the page it does not include any of the images.

    The code I am using is:

    curl_setopt ($ch, CURLOPT_URL, 'www.google.com' );
    curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
    $htmlPage = curl_exec ($ch);
    curl_close ($ch);
    echo $htmlPage

    Am I missing something here? Any help would be greatly appreciated.
     
    elambert, Feb 3, 2006 IP
  2. eiso

    eiso Peon

    Messages:
    583
    Likes Received:
    19
    Best Answers:
    0
    Trophy Points:
    0
    #2
    it's because the image links need to have http://www.google.com/ added before it, try using preg_match_all and then str_replace on the out put variable.
     
    eiso, Feb 3, 2006 IP
  3. drugoon

    drugoon Guest

    Messages:
    702
    Likes Received:
    10
    Best Answers:
    0
    Trophy Points:
    0
    #3
    you are getting only the html content of the page (like view source in IE or something) If the page contains images or external css files or flash files:these files are not recived through an CURL request. You can download them separately or you can point to them as eiso pointed it out
     
    drugoon, Feb 5, 2006 IP
  4. elambert

    elambert Peon

    Messages:
    2
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Thanks for your help.
     
    elambert, Feb 6, 2006 IP