Comparing 2 files for duplicates

Discussion in 'Programming' started by svinus, Dec 12, 2006.

0
  1. #1
    Hellow i have 2 files with in each file 40.000 lines with adresses. Now i want to compare this file and find adresses that are in both files (duplicates) and place this adresses in a 3th file.

    how can i do this easy ? (i don't have much experience with programming :) )
     
    svinus, Dec 12, 2006 IP
  2. crazybjörn

    crazybjörn Peon

    Messages:
    270
    Likes Received:
    10
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Hmm, one way would be to store the first file for example in a database and then compare the second file line for line against the database. Or you could just parse one of the files in an array and compare line by line against the other. But with 40,000 lines I'd recommend the DB option.
     
    crazybjörn, Dec 12, 2006 IP
  3. Barti1987

    Barti1987 Well-Known Member

    Messages:
    2,703
    Likes Received:
    115
    Best Answers:
    0
    Trophy Points:
    185
    #3 
    
//File 1
$file = fopen("file1.txt",'r');
while($info = fread($file,1027654)){
$total_a .= $info;
}
fclose($file);

//File 2
$file = fopen("file2.txt",'r');
while($info = fread($file,1027654)){
$total_b .= $info;
}
fclose($file);

$first = explode("\n",$total_a);
$second = explode("\n",$total_b);

$newadd = array();

foreach($first as $line){
 if(in_array($line,$second)){ array_push($newadd,$line."\n");
}

if(count($newadd) > 1){
//File 3
$file = fopen("file3.txt",'w');
frwite($file,$newadd);
fclose($file);
}
    PHP:
    Peace,
     
    Barti1987, Dec 13, 2006 IP