Hi. I have this code that one of the members of digital point so kindly made for me. I would like to make some changes to it, but since i have absolutely no clue about php, I was wondering if anyone here could help me. First, what the script does is simple. It goes to a url, and collects every megaupload link that is on that single page. The url is specified inside the file, and then they are all saved to a file, the name of which is also specified inside the file. What I would like it to do is first, I want to be able to put in many links. Like it would be a box, in which i can put as many links as i could possibly want. The urls are usually like this "www.sitename.com/2007/08/30/page-title.html What I want this to do is make a file for each link specified, and then name it after the last part, "page-title". So basically, I would paste a bunch of links into this box, and then press "submit" The script will start collecting links from each url, and then save it into a file with the last part of the url. That way, i won't have to edit the file everytime for a different link, since I have hundreds of these links. Here is the script: <?php function fetch_megaupload_links($url, $save_as = 'megaupload.txt') { if (!$fp = fopen($url, 'rb')) { trigger_error("Unable to open URL at {$url}", E_USER_ERROR); } $source = ''; while (!feof($fp)) { $source .= fread($fp, 8192); } fclose($fp); if (preg_match_all('/megaupload\.com\/\?d=([A-Z0-9]+)/i', $source, $links)) { $urls = array(); foreach ($links[1] AS $link) { $urls[] = "http://www.megaupload.com/?d={$link}"; } if (!$cache = @file($save_as)) { @touch($save_as); $cache = array(); } $fp = @fopen($save_as, 'w'); $urls = array_merge($urls, array_map('trim', $cache)); $stat = @fwrite($fp, implode("\n", array_unique($urls))); @fclose($fp); return (bool)$stat; } return false; } $url = 'http://www.anime-sensei.net/2006/05/hackgift.html'; var_dump(fetch_megaupload_links($url)); ?> PHP: I really hope someone can help me with this. Also, I understand that there could be a better way to do this with a different type of coding, so if that is possible, please tell me. Without this script, the site that I am developing won't go anywhere, so i really hope someone can help me. Thank You Nick
If you wrap the following around the code you already have it should work fine <?php $url_list = array('url1', 'url2', 'etc'); //the list of urls you want to visit foreach ($url_list as $url) { $regex = "|/([^/]+)\.html|"; $matches = preg_match($regex, $url, $results); if($matches) { $filename = $results[1]; //you may want to add .txt to the end fetch_megaupload_links($url, $filename); } else { //error handling //couldn't get filename } } ?> PHP:
Ok. When you say wrap, do you mean like this? <?php $url_list = array('url1', 'url2', 'etc'); //the list of urls you want to visit foreach ($url_list as $url) { $regex = "|/([^/]+)\.html|"; $matches = preg_match($regex, $url, $results); if($matches) { $filename = $results[1]; //you may want to add .txt to the end fetch_megaupload_links($url, $filename); } else { //error handling //couldn't get filename } } function fetch_megaupload_links($url, $save_as = 'megaupload.txt') { if (!$fp = fopen($url, 'rb')) { trigger_error("Unable to open URL at {$url}", E_USER_ERROR); } $source = ''; while (!feof($fp)) { $source .= fread($fp, 8192); } fclose($fp); if (preg_match_all('/megaupload\.com\/\?d=([A-Z0-9]+)/i', $source, $links)) { $urls = array(); foreach ($links[1] AS $link) { $urls[] = "http://www.megaupload.com/?d={$link}"; } if (!$cache = @file($save_as)) { @touch($save_as); $cache = array(); } $fp = @fopen($save_as, 'w'); $urls = array_merge($urls, array_map('trim', $cache)); $stat = @fwrite($fp, implode("\n", array_unique($urls))); @fclose($fp); return (bool)$stat; } return false; } $url = 'http://www.anime-sensei.net/2006/05/hackgift.html'; var_dump(fetch_megaupload_links($url)); ?> PHP:
Just about but you don't need the last couple of lines and you put the urls in at the top <?php $url_list = array('url1', 'url2', 'etc'); //the list of urls you want to visit foreach ($url_list as $url) { $regex = "|/([^/]+)\.html|"; $matches = preg_match($regex, $url, $results); if($matches) { $filename = $results[1]; //you may want to add .txt to the end fetch_megaupload_links($url, $filename); } else { //error handling //couldn't get filename } } function fetch_megaupload_links($url, $save_as = 'megaupload.txt') { if (!$fp = fopen($url, 'rb')) { trigger_error("Unable to open URL at {$url}", E_USER_ERROR); } $source = ''; while (!feof($fp)) { $source .= fread($fp, 8192); } fclose($fp); if (preg_match_all('/megaupload\.com\/\?d=([A-Z0-9]+)/i', $source, $links)) { $urls = array(); foreach ($links[1] AS $link) { $urls[] = "http://www.megaupload.com/?d={$link}"; } if (!$cache = @file($save_as)) { @touch($save_as); $cache = array(); } $fp = @fopen($save_as, 'w'); $urls = array_merge($urls, array_map('trim', $cache)); $stat = @fwrite($fp, implode("\n", array_unique($urls))); @fclose($fp); return (bool)$stat; } return false; } ?> PHP: You would need to populate the url_list array at the top. Are you comfortable with the html for a form? If you put your urls in a field separated by a space and then use explode it should be fairly quick but let me know if you need a hand.
Im not really sure what explode is, but yes, html would be good for a form. I am really not sure how to do it, so i would appreciate any help i can get. Thanks Nick
<html> <head> </head> <body> <?php //$url_list = array('url1', 'url2', 'etc'); //the list of urls you want to visit $url_list = explode(' ', $_POST['list']); foreach ($url_list as $url) { $regex = "|/([^/]+)\.html|"; $matches = preg_match($regex, $url, $results); if($matches) { $filename = $results[1]; //you may want to add .txt to the end var_dump(fetch_megaupload_links($url, $filename)); } else { //error handling //couldn't get filename } } function fetch_megaupload_links($url, $save_as = 'megaupload.txt') { if (!$fp = fopen($url, 'rb')) { trigger_error("Unable to open URL at {$url}", E_USER_ERROR); } $source = ''; while (!feof($fp)) { $source .= fread($fp, 8192); } fclose($fp); if (preg_match_all('/megaupload\.com\/\?d=([A-Z0-9]+)/i', $source, $links)) { $urls = array(); foreach ($links[1] AS $link) { $urls[] = "http://www.megaupload.com/?d={$link}"; } if (!$cache = @file($save_as)) { @touch($save_as); $cache = array(); } $fp = @fopen($save_as, 'w'); $urls = array_merge($urls, array_map('trim', $cache)); $stat = @fwrite($fp, implode("\n", array_unique($urls))); @fclose($fp); return (bool)$stat; } return false; } ?> <form action="" method="POST"> Type the urls you want to check separating each with a space. <textarea name="list"></textarea> <input type="submit" /> </form> </body> </html> PHP: Should do the trick.
It gives me these errors: Warning: fopen() [function.fopen]: HTTP request failed! HTTP/1.0 404 Not Found in /home/surf/public_html/anime/index.php on line 33 Warning: fopen(http://anime-sensei.blogspot.com/2006/05/hackgift.html http://anime-sensei.blogspot.com/2006/05/hack-legend-of-twilight-bracelet.html) [function.fopen]: failed to open stream: Permission denied in /home/surf/public_html/anime/index.php on line 33 Fatal error: Unable to open URL at http://anime-sensei.blogspot.com/2006/05/hackgift.html http://anime-sensei.blogspot.com/2006/05/hack-legend-of-twilight-bracelet.html in /home/surf/public_html/anime/index.php on line 35 Also, im not sure, but should this part be changed? function fetch_megaupload_links($url, $save_as = 'megaupload.txt') PHP:
I hadn't tested the code before I posted it but I have just done so now and it is working fine on my dev server. I suspect you may have allow_url_fopen disabled. Is it a server you run or does someone manage it for you?
We had this problem once. Try the code on your VDS. http://forums.digitalpoint.com/showthread.php?t=439504
Ok. I think it wasn't working with two urls because i was putting an enter between them instead of a space. The server is a vds, and I just checked to see that the fopen thing was on. When I run the script, it gives me a response, saying "bool(false)"
It should be returning bool(true). I take it the files aren't being created either? Are you still getting the errors or have they gone now?
nop the files arent creating. i tried putting in a megaupload.txt in root as well, and gave it 777 permissions, but it didnt help. http://surfoxy.com/anime/ I made the box bigger using a "style width and heigh, but i dont think that would affect it. it returns bool(false) for both, as long as there is only one space between each link. nico_swd, this is the script that worked on the vds, so im not sure why it's doing this now. :'(
Try running this. The messages returned are a little more informative. <html> <head> </head> <body> <?php //$url_list = array('url1', 'url2', 'etc'); //the list of urls you want to visit $url_list = explode(' ', $_POST['list']); foreach ($url_list as $url) { $regex = "|/([^/]+)\.html|"; $matches = preg_match($regex, $url, $results); if($matches) { $filename = $results[1]; //you may want to add .txt to the end var_dump(fetch_megaupload_links($url, $filename)); } else { //error handling //couldn't get filename } } function fetch_megaupload_links($url, $save_as = 'megaupload.txt') { if (!$fp = fopen($url, 'rb')) { trigger_error("Unable to open URL at {$url}", E_USER_ERROR); } $source = ''; while (!feof($fp)) { $source .= fread($fp, 8192); } fclose($fp); if (preg_match_all('/megaupload\.com\/\?d=([A-Z0-9]+)/i', $source, $links)) { $urls = array(); foreach ($links[1] AS $link) { $urls[] = "http://www.megaupload.com/?d={$link}"; } if (!$cache = @file($save_as)) { @touch($save_as); $cache = array(); } $fp = @fopen($save_as, 'w'); $urls = array_merge($urls, array_map('trim', $cache)); $stat = @fwrite($fp, implode("\n", array_unique($urls))); @fclose($fp); return "fwrite status:$stat"; } return "error parsing links"; } ?> <form action="" method="POST"> Type the urls you want to check separating each with a space. <textarea name="list"></textarea> <input type="submit" /> </form> </body> </html> PHP:
Okay so the problem is in writing the data to the local file. What are the permissions on the directory the file is in? It may be they are too strict.
Ok. it was on 755. I just changed it to 777, and I think it worked. The files were created. Thank you very much. Is the statues that it replies with the file size?
Oh and is it possible to have it done with enters instead of spaces? It gives an error when there is an enter between the links instead of a space.
Replace: $url_list = explode(' ', $_POST['list']); PHP: With: $url_list = preg_split('/\s+/', $_POST['list'], -1, PREG_SPLIT_NO_EMPTY); PHP: \s should match spaces and new lines. And just in case, you could replace var_dump(fetch_megaupload_links($url, $filename)); PHP: With var_dump(fetch_megaupload_links(trim($url), $filename)); PHP: To make sure there are no spaces in the URL.