Using PHP to retrieve parts of a file without opening the entire file

I was coding a simple php script using fopen() in order to search through file in a directory for selected strings, however this is impractical to search through server log files, as some are over 2GB in size (and thus would cripple my machine). I was wondering if there was a function (either built in or that someone had coded) that would allow me to retrieve parts a file without actually loading it all into memory first.

 

If you have PHP 5, you could try file_get_contents() and its offset and max length parameters.

http://www.php.net/file-get-contents

2 GBs is quite a lot though. Maybe you should look for other methods to achieve the same. Like a database and a custom error handle function.

 

Thanks, i'll look into it

yeah, 2GB is a large file size (luckily that's a month's traffic and a days, lol), however I only have read access to that server. besides, modifying files it out of the scope of the script, i want to keep it focused on reading files

 

Something like this will read the file one line at a time. You still have to scan the entire file, but the memory requirements are reasonable because you don't load it all at once:

<?php
$handle = @fopen("file.log", "r");
if ($handle) {
    while (!feof($handle)) {
        $buffer = fgets($handle, 4096);
        if (preg_match ("/search string/", $buffer)) {
          echo ("Match: $buffer");
        }
    }
    fclose($handle);
}
?>

PHP:

 

thans, that would be far nicer on my poor pc However, I thought fopen loaded the entire file into memory anyway, because it had to.

 

I tried to test that but the memory limit functions are not enabled in the PHP at my host.

It would be a huge flaw in PHP if you couldn't do this since it's a standard technique for processing large files. Do you have a link to some evidence?

 

Unfortunately not, it was what my lecturer told me back in college