removing duplicate entries in a list

I have a list of roughly 2300 keywords I need to sort through (ouch), and as they're simply copied from adwords there are a lot of duplicates. What Im needing is a quick script that'll take the list and remove any duplicate entries, only leaving the original one, eg

dog
dog
cat
mouse
fish
bird
Bush
chimp
mouse
rat
dog

becomes

dog
cat
mouse
fish
bird
rat

I was thinking of using two arrays, where all the keywords are stored in the first and looped through, adding it to the second if it doesn't already exist there, but that can be a bit of a resource-heavy so I was wondering if anyone knows of a quicker way, maybe something similar to array_intersect()?

 

i would put them all into one array then do an array sort. then delete the ones that are done twice.. heres an example.

	
        $query = "SELECT * FROM store ORDER BY 'country'";
	//note you would do a asort($array); instead
	$result = mysql_query($query) or die(mysql_error());
	$num_rows=mysql_num_rows($result);
	$x=0;	
	while($row = mysql_fetch_array($result))
	{
          if ($arrcountry[$x-1]!=$row['country']) {$arrcountry[$x]=$row['country']; ++$x;}
	}

PHP:

 

thanks for the help, buddy of mine modified ur script to my needs for me

 

no prob, i would have posted a better version of the script more applicable to your needs but didnt have time to look up the page i had the source on.

 

array_unique() is a php function that removes duplicate entries. To use it with what you're trying to do put the keywords in a text file, read in the contents of that text file (file_get_contents()), then explode() it into an array and use array_unique on it. Good luck!

 

shit didnt know about that.. it kicks ass

 

PHP is such a great language to play around with because it comes with so many goodies... One time I was working on a large project involving mucho arrays and just read every single array function in the manual. You have no idea how much candy they give you until you do that

end tangent

 

lol i know the feeling, its why i'm a php addict. My java skills have gone to shit cuz of the ease of php programming.