Parsing A CSV Question..

OK. I am parsing a CSV that gets updated regularly then thrown into a database.

Right now I have a script working as follows:

I initiate the script and it downloads to my server then it parses it into the database.

What I now am trying to do:
The next time I start the script I want it to find the number of new rows and only parse those. (to avoid duplication)

So what I need to do is:
Get the total number of current rows (100% complete)
Get the total number of TOTAL rows in the new CSV (100% complete)
Get the total number of new items by subtracting total from current. (100% complete)

I am stuck trying to get my script to do the correct thing, but I am so close.
Below is the current script:


$handle = fopen("csvdata/DYCData.csv", "r");
while (!feof($handle))
{
// grab one line of data
$data = fgets($handle);
// trim the newline from end of line
$data = trim($data);
// put fields into array separated by ," (comma, quote), this still leaves a trailing quote that we have to get rid of
$fields = explode(',"', $data);

// trim the trailing quote and echo the fields
for ($i=0; $i<sizeof($fields); $i++)
{
$fields[$i] = rtrim($fields[$i], '"');
$fields[$i] = ltrim($fields[$i], '"');
//echo 'fields['. $i . ']: ' . $fields[$i] . '<br />';
}

----------------------

Let's say the current number of rows is 100. And the updated CSV has 120 rows. That leaves 20 new items.

How can I have my loop start from row 1, and proceed through the loop until it gets to row 20?

I think it's something simple i need to add,...but I only get column modification errors and not any row changes.


Thank you for your help if you can.
-Tim

 

// Open handle to your Old CSV for comparison
$handle_old = file("csv/DYCData-OLD.csv");
$old_records = count($handle_old); // Get record count in old CSV

// Open handle to your New CSV
$handle = fopen("csvdata/DYCData.csv", "r");
while (!feof($handle))
{
// grab one line of data
$data = fgets($handle);
// trim the newline from end of line
$data = trim($data);
// put fields into array separated by ," (comma, quote), this still leaves a trailing quote that we have to get rid of
$fields = explode(',"', $data);

// Find new number of fields
$new_records = count($fields);
$total_new = $old_records - $new_records;

// trim the trailing quote and echo the fields
for ($i=0; $i<sizeof($fields); $i++)
{
// Check to make sure this is a new record, if so, trim and echo
if ($i <= $total_new) {
$fields[$i] = rtrim($fields[$i], '"');
$fields[$i] = ltrim($fields[$i], '"');
//echo 'fields['. $i . ']: ' . $fields[$i] . '<br />';
}
}

PHP:

Basically, what you can do is count the number of records in your OLD csv, compare it to the new csv, and only echo the difference.

 

I would also suggest you use fgetscsv or str_getcsv: (http://www.php.net/manual/en/function.fgetcsv.php) instead of trying to use your own parsing function.

 

I second the post above, don't even think about your own code when they have a perfectly working one...