Hello, I'm sick of the horrid script known as the CJ Datafeed. I'd greatly prefer to scrape X content out of CJ and put it into my own MySQL database. Does anyone have any recommendations? I've considered writing such a script, but I've never written a scraper before. Brandon
I wrote some php programs that I use to get the datafeed into my mysql database and then display them on pages. The CJ sends me datafeed everyweek and it is painful to re-run the same thing everyweek. I also do some customization (as the advertiser's categories sometimes are clobbered)..!! So I need to break them into cat -> subcat...!! You can checkout one of my sites (design is not good), where I parsed the datafeed from superjewelers. The site is http://jewelry.usadealsclub.com/ [Admin, I can remove the url from here, if it is considered spam] I would say, if u can find a cj scraper, please post here as we may all want that. I currently have a painful way of achieving the same thing to some extent. I am planning to sell (or license) the scripts at some later time..may be after June when it is more stable...!!
Sorry going a bit off-topic here: jazz, your site has some flaw: I clicked BUY button for "14k White Gold 1/4ct Diamond Pave Set Ring" (just to see what happens), so I saw the coupon 10% off discount tip - but when I it went to superJeweler site, the cart was empty maybe you're loosing some sales there. I went back (to your site) and clicked buy again and now the cart was updated with 2 blank items (no description, $0.00 price) - looks like it is showing the cart before add the item and after all added a blank item.
check mine laptop battery, I have srape... CJ Websevices too. to jazz7620, make hidden your buyUrl...avoid from from Googlebot, that your site is affilite site purpose.
Hi Micromag, I think the product that you clicked might have expired / discontinued..I have not updated the database in a while...one drawback when using the datafeed (which comes every week)...!! Hi pogung177, Could you please explain? What do u mean be hiding the buyUrl? and how will it affect googlebot? Please share if you know the trick.
Micromag & jazz7620, By hidden your buyUrl, it will avoid your page to supplemetal result on google. I have CJ site have indexed over 60k pages on Google by hidden all CJ url with redirect. Make sure the redirect links set to "nofollow."
No, your page is added to supplemental result when: You link to orphan pages (you link to pages that no one links) or your pages are orphan. You have copied content from another page (duplicated content) Have same title for all pages on your site or have meta tag problems There is no relation with supplemental result and using affiliate link on your site.
I have read on Matt Cutts blog including part of comment about this case http://www.mattcutts.com/blog/indexing-timeline/
I read the blog. I don't see anything about using datafeed URLs. He was speaking of a site that had NOTHING but affiliate content. Of course, if a site has NOTHING but affiliate content, that is not going to be new or incredibly useful to users. I guess it doesn't hurt to be careful. Awesome. I may start developing something like this soon, but let me know when you get your script finished. Brandon
I dont think so, but you can access Ebay products via their API. I have ever try it, with rss format so you only need Ebay API account to access. A lot of sample script on Google