I'm using cURL to login to an account and download .zip files automatically. I login just fine, but instead on downloading a file, I get back a "no JavaScript" page. I'd like to know if it is possible to get around it. The download links look like this: http://example.com/?download=1872&key=a6fc When I follow that link with CURL, I get this... <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> <meta http-equiv="Content-Script-Type" content="text/javascript"> <script type="text/javascript"> function getCookie(c_name) { // Local function for getting a cookie value if (document.cookie.length > 0) { c_start = document.cookie.indexOf(c_name + "="); if (c_start!=-1) { c_start=c_start + c_name.length + 1; c_end=document.cookie.indexOf(";", c_start); if (c_end==-1) c_end = document.cookie.length; return unescape(document.cookie.substring(c_start,c_end)); } } return ""; } function setCookie(c_name, value, expiredays) { // Local function for setting a value of a cookie var exdate = new Date(); exdate.setDate(exdate.getDate()+expiredays); document.cookie = c_name + "=" + escape(value) + ((expiredays==null) ? "" : ";expires=" + exdate.toGMTString()) + ";path=/"; } function getHostUri() { var loc = document.location; return loc.toString(); } setCookie('YPF8827340282Jdskjhfiw_92893745918266', '555.57.247.198', 10); try { location.reload(true); } catch (err1) { try { location.reload(); } catch (err2) { location.href = getHostUri(); } } </script></head><body> <noscript>This site requires JavaScript and Cookies to be enabled. Please change your browser settings or upgrade your browser.</noscript> </body></html> HTML: And the relevant part of the code I am using is... $ch = curl_init(); curl_setopt($ch,CURLOPT_URL,$FileURL); curl_setopt($ch,CURLOPT_RETURNTRANSFER,TRUE); curl_setopt($ch,CURLOPT_FOLLOWLOCATION,TRUE); curl_setopt($ch,CURLOPT_COOKIEJAR,$cookiefile); $fp = fopen($FinalFile,'w+'); curl_setopt($ch,CURLOPT_FILE,$fp); $data = curl_exec($ch); fclose($fp); curl_close($ch); PHP: Thank you in advance.
Sounds like you're trying to download from a site filled with scripttardery, so unless you have a way to actually run javascript on the server (like using a server-side browser engine) you are likely SoL on using curl to download from that site. Though you MIGHT be able to reverse engineer their script by doing a normal download from them and grabbing all the .js it uses along the way.