Some simple database functions

I thought these might be of interest to someone:

Feel free to employ and experiment.

<?php


/*
	
###############################   DATABASE I/O FUNCTIONS	#####################################
#												    
#	A library of Random Access Flat Text File I/O Functions for Database Applications in PHP    
#									         		    
#   ©2008 Chris Eccles	Please leave copyright text in file.
#                       Bug reports to me at: chris@kbodance.com									
#												
#	These routines run at least as fast as MySQL or Postgre Database back-ends but require
#   you to understand the structure of your files.
#
#   They are useful if you have no SQL database on your hosting, or if you simply want
#   to explore how multi-user databases and index files work.  Enjoy !
#
#	They utilise fixed-record-length file structures with multiple indexing possible on any key
#	field or group of fields.  Any primary file may have an unlimited number of index files, of	
#	either UNIQUE-key or DUPLICATE-key configuration. UNIQUE-key record extraction				
#	and DUPLICATE-key record extraction is by means of an ARRAY structure in memory.			
#	Longer keys will increase access time, but only arithmetically. Using a 12-digit key will	
#	enable file read rates of ca. 250-300 records/sec on a primary file of FRL<=250 bytes, on	
#	most decent Apache server systems.
#
#   Obviously, you have to decide how to assemble a line in the file to hold all your data
#   for a record, and you have to write the concatenation/padding routines in PHP yourself !
#   Do NOT pad with ‘@’ signs !  - these have a reserved use when deleting records.						
#      ===
#										
#	An index reconstruction function ['makeIndex'] is included for routine or emergency use.
#											
#	Functions:	writeToIndex		    adds a key to an index after appending a primary file	
#			    recnumFromIndex		    recovers a primary file record number from a UNIQUE index
#			    recRangeFromIndex	    recovers an array of primary file rec numbers from a	
#						                DUPLICATE-key index					
#			    writeRandom		        writes or appends a record to a primary file
#			    newkeyToIndex		    rewrites a changed primary file key to its index
#			    readRandom		        reads a record from a primary file			
#			    makeIndex		        rebuilds an index from a primary file			
#			    cleanFile		        produces a new primary file stripped of deletions	
#			    makeDualIndex		    rebuilds dual key index from primary file		
#			    recRangeFromDualIndex 	recovers an array of primary file rec numbers from a	
#						                DUPLICATE Dual-key index			
#										
#	All functions have indigenous locking on the files to which they have access.	
#	No external locking functions are required for multi-user accesses.			
#										
#	umask(0017) used on all writes
#												
#											
#	file: diskiolib.php		Version: 2.02			October 26 2008	04:50	
#												
#################################################################################################
	
*/


function writeToIndex($indexfile_, $indlock_, $key_, $recnum_) {

//## 	ONLY USE IF APPENDING TO A FILE
//##	===============================

		$indrecnum = "000000" . (strval($recnum_));
		$length = strlen($indrecnum);
		$finalrecnum = substr($indrecnum, ($length - 6), 6);
		$record = $key_ . $finalrecnum . "\n";		
		while 	(file_exists($indlock_)) {			// wait until file is free
			$dummy = NULL;															
		}
		
		umask(octdec("012"));
				
		$tempindlock =	fopen($indlock_ , "a") or exit ("CANT OPEN LOCK");
		fwrite($tempindlock, "L");
		fclose($tempindlock);
		
		umask(octdec("012"));
				
		$indpointer = fopen($indexfile_ , "a");		
		fwrite($indpointer, $record);
		fclose($indpointer);		
		unlink($indlock_);		
return;
}

function newkeyToIndex($indexfile_, $indlock_, $key_, $fileRecnum_, $indRecnum_) {

//## 	ONLY USE IF A KEY FIELD HAS CHANGED
//##	-----------------------------------
//##	$key_ MUST be passed with correct length for the index in question
//##	$fileRecnum_ and $indRecnum_ must be integer.
//##	This function is inefficient if used routinely, so avoid changing
//##	primary file keys where possible !  This is a bit of a "LOCK-HOG".
//##	==================================================================

		$newFileRecnum = "000000" . (strval($fileRecnum_));
		$length = strlen($newFileRecnum);
		$finalrecnum = substr($newFileRecnum, ($length - 6), 6);
		$record = ($key_ . $finalrecnum);		
		while 	(file_exists($indlock_)) {			// wait until file is free
			$dummy = NULL;															
		}
		
		umask(octdec("012"));
				
		$tempindlock =	fopen($indlock_ , "a") or exit ("CANT OPEN LOCK");
		fwrite($tempindlock, "L");
		fclose($tempindlock);		
		$workstring	=	file_get_contents($indexfile_);
		$workarray	=	explode("\n", $workstring);	
		$workarray[($indRecnum_ - 1)] = $record;
		$workstring	=	implode("\n", $workarray);
		
		umask(octdec("012"));
		
		$pointer	=	fopen($indexfile_, "w");
						fwrite($pointer, $workstring);
						fclose($pointer);	
		unlink($indlock_);					// free the file for another user		
return;
}


function recnumFromIndex($indexfile_, $indlock_, $key_, $keylen_) {

//##	this will only work for UNIQUE indexed files / returns the LAST match found
//##	===========================================================================

$recNum = 0;
if (file_exists($indexfile_)) {

		while 	(file_exists($indlock_)) {	// wait until file is free
		$dummy = NULL;															
		}
		
		umask(octdec("012"));
				
		$tempindlock =	fopen($indlock_ , "a") or exit ("CANT OPEN LOCK");
		fwrite($tempindlock, "L");
		fclose($tempindlock);		
		$workstring	=	file_get_contents($indexfile_);
		unlink($indlock_);		
		if ((strlen($workstring)) > 1) {		
			$workarray	=	explode("\n", $workstring);		
			foreach ($workarray as $key => $record) {		
				if (substr($record, 0, $keylen_) == $key_) {			
					$recNum = intval((substr($record, $keylen_, 6)));			
				}
			}	
		}						
}		
return($recNum);		
}


function recRangeFromIndex($indexfile_, $indlock_, $key_, $keylen_) {

//##	this will work for DUPLICATE key index files / returns an ARRAY of record
//##	numbers as $recNums or NULL if no records found.
//##	=========================================================================
		$recNums[] = NULL;
		
    if (file_exists($indexfile_)) {
    
		while 	(file_exists($indlock_)) {	// wait until file is free
		$dummy = NULL;															
		}
		
		umask(octdec("012"));
				
		$tempindlock =	fopen($indlock_ , "a") or exit ("CANT OPEN LOCK");
		fwrite($tempindlock, "L");
		fclose($tempindlock);		
		$workstring	=	file_get_contents($indexfile_);
		unlink($indlock_);
		
		if ((strlen($workstring)) > 1) {
				
			$workarray	=	explode("\n", $workstring);		
			foreach ($workarray as $key => $record) {
					
				if (substr($record, 0, $keylen_) == $key_) {
							
					array_push($recNums, intval(substr($record, $keylen_, 6)));
				}
			}	
		}		
    }
unset($recNums[0]);
return($recNums);	
}


function makeDualIndex($file_, $lock_, $indexfile_, $indlock_, $keypos1_, $keylen1_, $keypos2_, $keylen2_) {

//##	this creates a brand new 2-key index from a named datafile using ANY TWO chunks
//		of the FRL as the key fields.  NOTE THAT IT HOGS THE LOCKS while doing so.
//		Pass the position of the desired keys in $keypos_ and their lengths in $keylen_
//		Function returns $indexfile on success or "FAIL" if there is a problem.
//
//		FILES SHOULD BE CLEANED OF DELETIONS WITH FUNCTION: 'cleanFile()' BEFOREHAND.
//		=============================================================================

$data_ = "FAIL";
$countrec = 0;
		while 	((file_exists($lock_)) || (file_exists($indlock_))) {
		$dummy = NULL;															
		}
		
		umask(octdec("012"));
		
		$templock =	fopen($lock_ , "a") or exit ("CANT OPEN LOCK");
		fwrite($templock, "L");
		fclose($templock);
		
		umask(octdec("012"));
				
		$tempindlock =	fopen($indlock_ , "a") or exit ("CANT OPEN LOCK");
		fwrite($tempindlock, "L");
		fclose($tempindlock);
						
		$pointer 	= fopen($file_ , "r");
		
		umask(octdec("012"));
		
		$indpointer = fopen($indexfile_ , "w");		
			while	(!feof($pointer)) {
				$workstring = fgets($pointer);		
				$newkey1 = substr($workstring, $keypos1_, $keylen1_);					
				$newkey2 = substr($workstring, $keypos2_, $keylen2_);				

					if 	((strlen($newkey1) > 0) and (strlen($newkey2) > 0)) {
										
					$countrec = $countrec + 1;		
					$indrecnum = "000000" . (strval($countrec));
					$length = strlen($indrecnum);		
					$finalrecnum = substr($indrecnum, ($length - 6), 6);
					
					$record = $newkey1.$newkey2.$finalrecnum . "\n";
					
					$oldMask = umask(0017);
							
					fwrite($indpointer, $record);					
					}
			}
		fclose($pointer);
		fclose($indpointer);
		unlink($lock_);
		unlink($indlock_);
		$data_ = $indexfile_;
return($data_);
}

function recRangeFromDualIndex($indexfile_, $indlock_, $primKey_, $primKeylen_, $secKeylen_, $secKeyLower_, $secKeyUpper_) {

//##	this will work for DUPLICATE 2-key index files / returns an ARRAY of record
//		numbers which contain $primKey_ and which also contain $secKey_ in range specified by $secKeyLower_
//		> $secKeyUpper_  by arithmetic evaluation.  Index file requires to have been constructed
//		using function makeDualIndex (see above), or appended using function writeToIndex, provided that
//		the two keys were concatenated to produce a single string and $keylen_ adjusted accordingly.
//##	===================================================================================================
		$recNums[] 	= NULL;
		$recnumPos	= ($primKeylen_ + $secKeylen_);
			
if (file_exists($indexfile_)) {

		while 	(file_exists($indlock_)) {	// wait until file is free
		$dummy = NULL;															
		}
		
		umask(octdec("012"));
				
		$tempindlock =	fopen($indlock_ , "a") or exit ("CANT OPEN LOCK");
		fwrite($tempindlock, "L");
		fclose($tempindlock);
				
		$workstring	=	file_get_contents($indexfile_);
		unlink($indlock_);
				
		if ((strlen($workstring)) > 1) {
				
			$workarray	=	explode("\n", $workstring);
				
			foreach ($workarray as $key => $record) {
			
				$primWork	= (substr($record, 0, $primKeylen_));
				$secWork	= (substr($record, $primKeylen_, $secKeylen_));
						
				if (($primWork == $primKey_) and ($secWork >= $secKeyLower_) and ($secWork <= $secKeyUpper_)) {			
					array_push($recNums, intval(substr($record, $recnumPos, 6)));
				}
			}	
		}		
}
unset($recNums[0]);
return($recNums);	
}


function cleanFile($file_, $lock_) {

//## 	cleans file of deleted records flagged as all "@"
//##	returns number of records written to new file
//
//		NOTE: this is a "LOCK-HOG". Do not use routinely
//		but only as part of a cron maintenance schedule.		
//		=================================================

$countrec = 0;
$compare = "@@@@@@@@";
while 	((file_exists($lock_)) or (file_exists("data/temclock.txt"))) {	// wait for BOTH files free
		$dummy = NULL;															
		}
		
		umask(octdec("012"));
				
		$templock =	fopen($lock_ , "a") or exit ("CANT OPEN LOCK");
		fwrite($templock, "L");
		fclose($templock);
		
		umask(octdec("012"));
				
		$templock2 =	fopen("data/temclock.txt", "a") or exit ("CANT OPEN LOCK");
		fwrite($templock2, "L");
		fclose($templock2);
		$pointerOld 	= fopen($file_ , "r");
		
		umask(octdec("012"));
		
		$pointerNew 	= fopen("data/tempfile", "w");
		while	(!feof($pointerOld)) {			
				$workstring = fgets($pointerOld);				
				rtrim($workstring);				
				if (substr($workstring, 0, 8) != $compare) {
				
				umask(octdec("012"));
						
				fwrite($pointerNew, $workstring);
				$countrec = $countrec + 1;
				}									
		}		
		fclose($pointerOld);
		fclose($pointerNew);		
		unlink($file_);
		
		umask(octdec("012"));
				
		rename("data/tempfile", $file_);		
		unlink($lock_);
		unlink("data/temclock.txt");
		
return($countrec);

}


function makeIndex($file_, $lock_, $indexfile_, $indlock_, $keypos_, $keylen_) {

//##	this creates a brand new index from a named datafile using ANY chunk
//		of the FRL as the key field.  NOTE THAT IT HOGS THE LOCKS while doing so.
//		Pass the position of the desired key in $keypos_ and its length in $keylen_
//		Function returns $indexfile on success or "FAIL" if there is a problem.
//
//		FILES SHOULD BE CLEANED OF DELETIONS WITH FUNCTION: 'cleanFile()' BEFOREHAND.
//		=============================================================================

$data_ = "FAIL";
$countrec = 0;
		while 	((file_exists($lock_)) || (file_exists($indlock_))) {	// wait for BOTH files free
		$dummy = NULL;															
		}
		
		umask(octdec("012"));
		
		$templock =	fopen($lock_ , "a") or exit ("CANT OPEN LOCK");
		fwrite($templock, "L");
		fclose($templock);
		
		umask(octdec("012"));
				
		$tempindlock =	fopen($indlock_ , "a") or exit ("CANT OPEN LOCK");
		fwrite($tempindlock, "L");
		fclose($tempindlock);		
		$pointer 	= fopen($file_ , "r");
		
		umask(octdec("012"));
		
		$indpointer = fopen($indexfile_ , "w");		
			while	(!feof($pointer)) {
				$workstring = fgets($pointer);		
				$newkey = substr($workstring, $keypos_, $keylen_);				
				rtrim($newkey);		
					if 	(strlen($newkey) > 0) {					
					$countrec = $countrec + 1;		
					$indrecnum = "000000" . (strval($countrec));
					$length = strlen($indrecnum);		
					$finalrecnum = substr($indrecnum, ($length - 6), 6);
					$record = $newkey . $finalrecnum . "\n";
					
					umask(octdec("012"));
							
					fwrite($indpointer, $record);					
					}
			}
		fclose($pointer);
		fclose($indpointer);
		unlink($lock_);
		unlink($indlock_);
		$data_ = $indexfile_;
return($data_);
}


function writeRandom($file_, $lock_, $data_, $recnum_, $reclen_) {

//##	$data_ MUST BE ($reclen_ - 1) in size AND PADDED WITH %A0 TO PRESERVE INTEGRITY OF FILE
//##
//##	if $recnum_ is passed as '0', this function APPENDS to the end of $file_
//##
//## 	NOTE: this function does NOT update index files but DOES return the record number written
//##	-----------------------------------------------------------------------------------------
//##
//##	the function 'writeToIndex' requires to be invoked to ADD
//##	a record to each index if it is APPENDED to the main file here.
//##
//##	if the new data content changes the key field of any index, the index
//##	file(s) will require to be rewritten with 'newkeyToIndex' to reflect this change.
//##	=========================================================================================

while 	(file_exists($lock_)) {						// wait until file is free
		$dummy = NULL;				
		}
		
		umask(octdec("012"));
				
		$tempLock =	fopen($lock_, "a") or exit ("CANT OPEN LOCK");
		fwrite($templock, "L");
		fclose($templock);
if	($recnum_ < 1) {
		
		$pointer = fopen($file_, "a");
		
		umask(octdec("012"));
		
		fwrite($pointer, $data_ . "\n");
		$size = fstat($pointer);
		$recnum_ = ($size[size] / $reclen_);		// gives effective record number after an APPEND
		fclose($pointer);
		unlink($lock_);								// free the file for another user
}		
else {		
		$workstring	=	file_get_contents($file_);
		$workarray	=	explode("\n", $workstring);	
		$workarray[($recnum_ - 1)] = $data_;
		$workstring	=	implode("\n", $workarray);
		$pointer	=	fopen($file_, "w");
		
		umask(octdec("012"));
		
						fwrite($pointer, $workstring);
						fclose($pointer);	
		unlink($lock_);								// free the file for another user
}	
return($recnum_);		
}


function readRandom($file_, $lock_, $recnum_, $reclen_) {

//##	RECORDS MUST BE CORRECT LENGTH AND PADDED WITH %A0 TO ENSURE INTEGRITY OF READ
//##
//##	Returns $data_ as STRIPPED string of length ($reclen_ -1); ie: no NEWLINE char
//##
//##	$recnum_ must be provided from function 'recnumFromIndex' 
//##	BEFORE calling 'readRandom'
//##	==============================================================================

$data_ = NULL;
if (file_exists($file_)) {
	while 	(file_exists($lock_)) {					// wait until file is free
		$dummy = NULL;				
		}
		
		umask(octdec("012"));
			
		$tempLock =	fopen($lock_, "a") or exit ("CANT OPEN LOCK");
		fwrite($templock, "L");
		fclose($templock);						
		$pointer = fopen($file_, "r");
		$offset = (($recnum_ - 1) * $reclen_);		
		$size = fstat($pointer);		
		if ((($size[size]) / $reclen_) >= $recnum_) {		
		rewind($pointer);
		fseek($pointer, $offset);		
		$workstring = fgets($pointer);
		fclose($pointer);		
		unlink($lock_);								// free the file for another user
		$data_ = rtrim($workstring);		
	}		
else {		
		$data_ = NULL; 								// attempt to read past end of file		
}
}					
return($data_);			
}

?>

Code (markup):

Cheers.

Chris

 

Great Work

 

Thanks, yoursanjay.

The routines are not blessed with any sophisticated error trapping, and you DO have
to be careful that you map out your record structure on squared paper as a cheat-sheet when
designing around these functions, but they have the advantage of utter simplicity.

I have done three shopping carts using these routines and had no problems.

If your data files or indexes do need attention, you can either write a simple admin web page or even
just download them, do a fix in BBE, and upload them again.

Chris