best way to store data

Hi

Can someone tell me what the best way to store data locally with php is. I want to use a flat file rather than a normal database.

The data I will be storing is basicly

1000's of keyword and urls and also some other data that determines when the data was last updated. I would imagine storing this data in a normal txt file could be very slow, I need it to be as fast as possible really.

Any help with be appreciated as I have no idea where to start with this.


Thanks
Matthew Keen

 

MySQL and PHP are the ticket. Blazingly fast stuff.

 

yep you should definatly put this stuff in a database of some kind. MySQL is the most widely used so you should probably stick with it. VERY easy to integrate mysql and php.

 

Thanks for the reply guys but please please please

Read what I said before answering.


Thanks

Matthew

 

Well then, let's see.

You need PHP and MYSQL. You should know HTML, Preferably XHTML.

You need to define the data you will collect and make tables to store it with a place for each field of data in records.

You can define your DB in phpmyadmin, however, you will probably be writing php and mysql and html code by hand to manage and use the data.

 

ok, then why do you want to use a flat file instead of a db?

 

@mkeen:
you probably do not have mysql on your server (get it, or change hosting co), but for the time being:

pick a (any) character that's not in either one of your fields (say TAB), do something like:

$data = array();

$fp = fopen("data", "r");
if ($fp)
{
  while(!feof($fp))
  {
    $buf = fgets($fp);
    if (strlen($buf) > 0)
    {
      $data[] = explode("\t", $buf);
    }
  }
  fclose($fp);
}

print_r($data);

PHP:

create a file called "data", fields should be TAB seperated
(didn't test this, so please fix syntax errors yourself)

 

Thanks for the replies everyone, I did clearly say "I want to use a flat file rather than a normal database."

I have now used something similar to what sarahk said, Im using a directory called urldata/ and storing each seperate record as a new file $url.txt in that directory, its working out very qell and is blazing fast because it only has to load one small file at a time rather than 1 large file storing 1000's of records.


Thanks
Matt

 

Just out of sick curiousity, why flat files as opposed to a dbms?

 

I always find the use of the term flat file as oppsed to a relational db odd, since most relational dbs people then talk about still are using the file systems (there are some that dont but usually the same people that talk in terms of 'flat files' don't have a clue of the impliementaion details).

Anyway, show us the output of your phpinfo() and we may be able to make better suggestions, but off hand if it is available, take a look at http://us2.php.net/manual/en/ref.dba.php some of these hash and isam libraries can blow the doors of mysql for the right problem sets.

Another option, though it is a db, it just uses a single file in your area to store the data, is sqlite.

 

PHP4 will allow .xml if the columns and structure are going to change in future

PHP 5 bundles SQLite which allows you to perform SQL statement on Flat-Files !

 

well not exactly, there are some perl modules that do that. sqlite just stores the db in a locally opened single file. On the other hand mysql uses client server, but in the back end, it is storing the data in files on the filesystems.

The term 'flat-files' has no real tech. meaning.

 

Reason im going with flat files is that the code I am writing is for a small advertising network, the user installs the files on his server and each url is assigned a random url, the url&anchor is requested from my server and then stored locally, I dont want a database because not everyone will have access to one.

 

another option for you then, since you are just doing random data access is fixed length records, and doing an fseek to the len*recno

 

MySQL or Plain-text Database are the choices.

 

How many records will be in this file?

Looping through and entire file with regex's for search is really poor code. You won't be doing anyone any favors with a large file. I see no reason why a site worth advertising can't invest in the extra dollar for a database. :/

Anywho, this could be ok for a small file. I lost track, where are you stuck?

 

The reason you lost track is because you dont seem to be reading all of the post, are you just scanning over it? What I was asking was what the best way to store data *>>without<<* a database (like mysql etc) was. Also as I have allready said, Im not looping through an entire file with regex, As I have allready stated each record is now stored in its own file.

In the following structure.

linksdata/
linksdata/url.com.txt
linksdata/url.com-sub.txt
linksdata/url.com-sub2.txt
linksdata/url.com-page1.txt
linksdata/url.com-page2.txt

Etc etc etc, only one file is accessed and the file only holds one small record.

Anyway as Ive allready said I have sorted this problem now.

 

A good way is to serialize $var into string,then store in flat file.
step1:
$s=serialize($you_wanted_store_var);
step2:
$fp=fopen($CFG[scriptDir]."/cache/$name.txt",'w');
fputs($fp,$s);
fclose($fp);

Restore from flat files:
$fp=@fopen($CFG[scriptDir]."/cache/$name.txt",'r');
$content=@fread($fp,@filesize($CFG[scriptDir]."/cache/$name.txt"));
$you_wanted_store_var=@unserialize( $content );