What's the best way to parse product (XML) feeds?

Hi there,

We would like to import product feeds (XML) from various affiliate networks in our database on a daily basis. What’s the best way to do this?

We were thinking of downloading the feeds (some are around 50mb) on our server each day with a cronjob. Then we can parse them and store the data in a mysql db.

Well parsing these feed isn’t going easy. Can you guys recommend me a good parser (free or paid) that is capable of parsing product feeds? (there are enough parsers for rss news feeds, but we are looking for an xml product feeds parser)

PS: I just posted this message a few minutes ago and somehow it disappeared or got deleted.

 

You can build a parser from the examples found here:

http://us3.php.net/manual/en/book.xml.php

 

How complicated are the actual feeds?

I would definitely get them locally on the server, and then work with them from there. How difficult this is going to be is really dependent on the complexity of the feed.

If they're fairly simple, and standard. Something like:

<product>
<title />
<url />
<price />
</product>

You could use simple XML to easily parse them.

One problem you may run into, unless you can figure out how to read line-by-line, is that you will need at least the amount of RAM that the file is to parse the script. Also, if you set any variables while the script is looping, the RAM usage will go up considerably. I've gotten php to parse GIG plus files, but even at 50Mb it may take some testing and tuning to get them to process cleanly.

 

Thanks for your feedback guys. I appreciate it

 

If you are not interested in the whole feed, but certain tags, preg_match_all works faster.

 

Using simple_xml ?

 

For large feeds you'll probably find The XMLReader class to be the best option.

Take note of this comment though.

Here's a nice article giving a run down of different parsing methods available:
XML for PHP Developers

kamm...