Templates for large CJ data feed?

Can anyone recommend a template/program for large CJ data feeds?

 

I'm not sure what you are asking. First thing, you have to pay $200 for an integration fee to have access to any merchant's datafeed in CJ. Once you get it, it can be delivered any way (I get mine through FTP).

 

Obviously tnomal has paid for datafeed. His problem may be how to unzip 2GB website or how to manage data of advertiser with several million products. The control panel of my website extracts zip files upto 2 GB. I use my script split.php to split large data into 50,000 product files shopcom.txt, shopcom1.txt..... shopcom38.txt

 

my data feed is just 3mb, but i'm considering adding more merchants once I get the hang of it.

Thing is, I get the feed as csv but saved in a .txt text file.

I then struggle to open it with notepad and save as csv which can then be manipulated with excel before being imported to mysql.

To handle 2gb, you must have a far better process, do you run your own web server , therefore direct physical access to the server

 

.csv and .txt are interchangeable - just rename the file. During 12 years I used MySQL 4 times - every time I required self learning for an hour. I think in Admin you have a feature wherein you feed text file and get data in MySQL. I have done that every time and never used excel. I have multiple accounts 350-2000 GB, with billions of web pages. buy.com, shop.com and some others have over 10 GB unzipped data - fortunately in multiple smaller files. This month I used largest one (1-2 GB) and discarded the rest.

 

Sorry, in excel .txt and .csv may not be used same way. I do not know excel. Also I use .txt files as data - No MySql.