I'm looking for a php script that would allow me to transfer files from a download area like mega upload, rapidshare, or other linked download etc. to my server without having to download the file and then upload it. I'm going to go ahead and try to write one, but if anyone knows of an existing one it would save me alot of trouble.
$data = file_get_contents("http://www.remote/file.zip"); $fp = fopen("file.zip", "wb"); fwrite($fp, $data); fclose($fp); PHP:
and if you want to use other than php you may use wget login to recipient server via telnet or SSH and use wget with correct options to either download server to server one or many files or complete folder ... from a download site directly to your server of course such also works with crontab if you want to regularly update a particular file or files - like making a mirror of certain files on your site
This does have the draw back that the whole file is read into ram, to then just write it out to disk. May be better to read the file in say 32kb at a time and write that out to disk incrementaly.
Thanks for the input guys, I can't use telnet for it. As for the php code, that's a very simplified vers of what I'd like to do but it involves a bit more than that to harvest from servers like megaupload or ftp. I'd also like it to have a clean web interface with some options like proxying the downloads, or uploading it to an email inbox straight from the source (gmail's inbox space just keeps growing ). I've gotten a pretty good start on it, any input would be apreciated. Just finishing the variables, and starting in on the meat of it ,lol I figure it'll take a couple more pages to finish, I don't have a whole lot of free time so if anyone wants to add onto it feel free to do so and post it. Just put any added lines with a **/-->added by: and your piece. And you can have you credits in there if there's any distribution ( open source of course ) Unless you feel like adding alot of code to it, then I'd be more than happy to start a collaboration with someone, and you can have your say as to how it's distributed. Here's how it's going so far: $handler : Object Handler $use_resume : use section download $use_autoexit : auto stop after finishing download $use_auth : use authentication download $data : Download Data $data_type : Download Data Type $data_mod : Last modified time $data_len : Download Data Len $filename : Download File Name $mime : File mime $seek_start : Start Seek $seek_end : End Seek $bufsize : BUFFER SIZE **/ var $handler = array('auth' => false ,'header' => false,'fopen'=>false,'fclose'=>false,'fread'=>false,'fseek' => false); var $use_resume = true; var $use_autoexit = true; var $use_auth = false; var $data_len = 0; var $data = null; var $data_mod = 0; var $filename = null; var $mime = null; var $bufsize = 2048; var $seek_start = 0; var $seek_end = -1; pre_download() : Pre Download Function download() : Download all file set_byfile() : Set data download by file set_bydata() : Set data download by data set_byurl() : Set data download by url set_filename() : Set file name set_mime() : Set file mime download_header() : Send header download_ex() : Manual Download **/ function pre_download() { global $HTTP_SERVER_VARS; if ($this->use_auth) { //use authentication if (!$this->_auth()) { //no authentication $this->_header('WWW-Authenticate: Basic realm="U:P"'); $this->_header('HTTP/1.0 401 Unauthorized'); $this->_header('status: 401 Unauthorized'); if ($this->use_autoexit) exit(); return false; } } if ($this->mime == null) $this->mime = "application/octet-stream"; //default mime if (isset($_SERVER['HTTP_RANGE']) || isset($HTTP_SERVER_VARS['HTTP_RANGE'])) { if (isset($HTTP_SERVER_VARS['HTTP_RANGE'])) $seek_range = substr($HTTP_SERVER_VARS['HTTP_RANGE'] , strlen('bytes=')); else $seek_range = substr($_SERVER['HTTP_RANGE'] , strlen('bytes=')); $range = explode('-',$seek_range); if ($range[0] > 0) { $this->seek_start = intval($range[0]); } if ($range[1] > 0) $this->seek_end = intval($range[1]); else $this->seek_end = -1; } else { $this->seek_start = 0; $this->seek_end = -1; } if ($this->seek_start < 0 || !$this->use_resume) $this->seek_start = 0; // echo $this->seek_start."-".$this->seek_end; return true; } function download_ex($size) { if (!$this->pre_download()) return false; ignore_user_abort(true); @set_time_limit(0); //Use seek end here if ($this->seek_start > ($size - 1)) $this->seek_start = 0; if ($this->seek_end <= 0) $this->seek_end = $size - 1; $this->download_header($size,$this->seek_start,$this->seek_end); $this->data_mod = time(); return true; } function download() { if (!$this->pre_download()) return false; $seek = $this->seek_start; ignore_user_abort(true); @set_time_limit(0); $size = $this->data_len; if ($this->data_type == 0) { $size = filesize($this->data); if ($seek > ($size - 1)) $seek = 0; if ($this->filename == null) $this->filename = basename($this->data); $res =& $this->_fopen($this->data,'rb'); if ($seek) $this->_fseek($res , $seek); if ($this->seek_end < $seek) $this->seek_end = $size - 1; $this->download_header($size,$seek,$this->seek_end); //always use the last seek $size = $this->seek_end - $seek + 1; while (!connection_aborted() && $size > 0) { if ($size < $this->bufsize) echo $this->_fread($res , $size); else echo $this->_fread($res , $this->bufsize); $size -= $this->bufsize; } $this->_fclose($res); } else if ($this->data_type == 1) { if ($seek > ($size - 1)) $seek = 0; if ($this->seek_end < $seek) $this->seek_end = $this->data_len - 1; $this->data = substr($this->data , $seek , $this->seek_end - $seek + 1); if ($this->filename == null) $this->filename = time(); $size = strlen($this->data); $this->download_header($this->data_len,$seek,$this->seek_end); while (!connection_aborted() && $size > 0) { echo substr($this->data , 0 , $this->bufsize); $this->data = substr($this->data , $this->bufsize); $size -= $this->bufsize; } } else if ($this->data_type == 2) { //just send a redirect header header('location : ' . $this->data); } if ($this->use_autoexit) exit(); return true; } function download_header($size,$seek_start=null,$seek_end=null) { $this->_header('Content-type: ' . $this->mime); $this->_header('Content-Disposition: attachment; filename="' . $this->filename . '"'); $this->_header('Last-Modified: ' . date('D, d M Y H:i:s \G\M\T' , $this->data_mod)); if ($seek_start && $this->use_resume) { $this->_header("Content-Length: " . ($seek_end - $seek_start + 1)); $this->_header('Accept-Ranges: bytes'); $this->_header("HTTP/1.0 206 Partial Content"); $this->_header("status: 206 Partial Content"); $this->_header("Content-Range: bytes $seek_start-$seek_end/$size"); } else { $this->_header("Content-Length: $size"); } } function set_byfile($dir) { if (is_readable($dir) && is_file($dir)) { $this->data_len = 0; $this->data = $dir; $this->data_type = 0; $this->data_mod = filemtime($dir); return true; } else return false; } function set_bydata($data) { if ($data == '') return false; $this->data = $data; $this->data_len = strlen($data); $this->data_type = 1; $this->data_mod = time(); return true; } function set_byurl($data) { $this->data = $data; $this->data_len = 0; $this->data_type = 2; return true; } function set_filename($filename) { $this->filename = $filename; } function set_mime($mime) { $this->mime = $mime; } function set_lastmodtime($time) { $time = intval($time); if ($time <= 0) $time = time(); $this->data_mod = $time; } PHP:
here u go <?php //get file @exec("wget http://example.com/file.zip"); //move file //change permissions to read only ?> PHP:
if you have a shell account on both machines, scp will let you copy between the two servers as if the locations were local. http://en.wikipedia.org/wiki/Secure_copy
unfortunately, rapidshare, megaupload etc. does not allow auto copying of files from the server. Also, they identify your IP and place download limits. I am too searching for a solution if it comes through.