1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Curl script not working on host

Discussion in 'PHP' started by Uruviel, Feb 15, 2010.

  1. #1
    I made a curl script to get the html from a page and parse it. On my laptop it works fine, but once uploaded to my host it doesn't work. My host has curl installed and the script works when I use a different url like www.google.be

    It gives me no errors and I have no clue what's going on on my host so that the script isn't working.

    
    <?php
    error_reporting(E_ALL);
    ini_set("display_errors", 1); 
    $target_url = "http://landofconfusion.freedkp.org/frx/listmembers.php?show=all";
    //$target_url = "http://www.google.be";
    
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL,$target_url);
    curl_setopt($ch, CURLOPT_FAILONERROR, true);
    curl_setopt($ch, CURLOPT_AUTOREFERER, true);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER,true);
    curl_setopt($ch, CURLOPT_TIMEOUT, 10);
    curl_setopt($ch, CURLOPT_COOKIEJAR, "my_cookies.txt");
    curl_setopt($ch, CURLOPT_COOKIEFILE, "my_cookies.txt");
    $html = curl_exec($ch);
    if (!$html) {
    	echo "cURL error number:" .curl_errno($ch);
    	echo "cURL error:" . curl_error($ch);
    }
    var_dump($html);
    PHP:
    The result:
    cURL error number:0cURL error:
    string(0) ""

    That doesn't give me much to work with sadly.
    Any idea what I can do so I get at least an error out of it so I can resolve whatever issue there is. Or maybe somebody experienced this before. Any help is welcome :)
     
    Uruviel, Feb 15, 2010 IP
  2. Kalyse

    Kalyse Peon

    Messages:
    1,221
    Likes Received:
    24
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Could be an issue that because you are trying to access a privileged page, the headers which are sent is a permission denied.
    I don't know how cURL would react, but you should check CURLOPT_HEADER and see!
     
    Kalyse, Feb 15, 2010 IP
  3. Uruviel

    Uruviel Peon

    Messages:
    5
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    the result when I add the CURLOPT_HEADER:

    string(427) "HTTP/1.1 302 Found Date: Mon, 15 Feb 2010 13:10:05 GMT Server: Apache Set-Cookie: herefirst=yes Set-Cookie: PHPSESSID=db2bd56fbe68ace84f7a0a6c98d922bd; path=/ Expires: Thu, 19 Nov 1981 08:52:00 GMT Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Pragma: no-cache location: http://landofconfusion.freedkp.org Content-Length: 0 Connection: close Content-Type: text/html; charset=UTF-8 "
     
    Uruviel, Feb 15, 2010 IP
  4. szalinski

    szalinski Peon

    Messages:
    341
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    0
    #4
    use CURLOPT_FOLLOWLOCATION
     
    szalinski, Feb 15, 2010 IP
  5. Uruviel

    Uruviel Peon

    Messages:
    5
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Can't use that option because the host has an open_basedir set.
    I get the following error:
    Warning: curl_setopt() [function.curl-setopt]: CURLOPT_FOLLOWLOCATION cannot be activated when in safe_mode or an open_basedir is set
     
    Uruviel, Feb 16, 2010 IP
  6. szalinski

    szalinski Peon

    Messages:
    341
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    0
    #6
    then you'll have to find a way around it, try preg_match'ing the location header and then load that into curl. i think there's a function to do it somewhere on php.net mentioning your open_basedir problem, probably under curl_setopt.
     
    szalinski, Feb 16, 2010 IP
  7. Uruviel

    Uruviel Peon

    Messages:
    5
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #7
    I've found and tried that script. I think I'm having issues with setting and reading a cookie on my host. Though, I can open the cookie file with write permissions in the php script.

    Is there any other way to achieve this without using curl?


    Current script:
    
    <?php
    error_reporting(E_ALL);
    ini_set("display_errors", 1);
    
    function curl_redirect_exec($ch, &$redirects, $curlopt_header = false) {
        curl_setopt($ch, CURLOPT_HEADER, true);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
        $data = curl_exec($ch);
        $http_code = curl_getinfo($ch, CURLINFO_HTTP_CODE);
        if ($http_code == 301 || $http_code == 302) {
            var_dump($data);
            list($header) = explode("\n\n", $data, 2);
            $matches = array();
            preg_match('/(Location:|URI:)(.*?)\n/', $header, $matches);
            $url = trim(array_pop($matches));
            $url_parsed = parse_url($url);
            if (isset($url_parsed)) {
                curl_setopt($ch, CURLOPT_URL, $url);
                $redirects++;
                return curl_redirect_exec($ch, $redirects);
            }
        }
        if ($curlopt_header)
            return $data;
        else {
            var_dump($data);
            list(,$body) = explode("\n\n", $data, 2);
            return $body;
        }
    }
     
    $target_url = "http://landofconfusion.freedkp.org/frx/listmembers.php?show=all";
    //$target_url = "http://www.google.be";
    
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL,$target_url);
    curl_setopt($ch, CURLOPT_FAILONERROR, true);
    curl_setopt($ch, CURLOPT_AUTOREFERER, true);
    curl_setopt($ch, CURLOPT_TIMEOUT, 10);
    curl_setopt($ch, CURLOPT_COOKIEJAR, "my_cookies.txt");
    curl_setopt($ch, CURLOPT_COOKIEFILE, "my_cookies.txt");
    $redirects = 20;
    var_dump(curl_redirect_exec($ch, $redirects));
    
    PHP:
    Result:
    
    string(427) "HTTP/1.1 302 Found
    Date: Wed, 17 Feb 2010 12:17:04 GMT
    Server: Apache
    Set-Cookie: herefirst=yes
    Set-Cookie: PHPSESSID=701af3d2610f742ebed0d0912b6d2bc1; path=/
    Expires: Thu, 19 Nov 1981 08:52:00 GMT
    Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
    Pragma: no-cache
    location: http://landofconfusion.freedkp.org
    Content-Length: 0
    Connection: close
    Content-Type: text/html; charset=UTF-8
    
    "
    bool(false)
    <br />
    <b>Notice</b>:  Undefined offset:  1 in <b>/home/uruviel/domains/uruviel.be/public_html/test.php</b> on line <b>27</b><br />
    NULL
    
    
    PHP:
     
    Last edited: Feb 17, 2010
    Uruviel, Feb 17, 2010 IP