Advise Needed! How to extract data from amazon?

Discussion in 'Programming' started by siwarin.e, Aug 8, 2008.

  1. #1
    http://www.amazon.com/s/qid=1218207428/ref=sr_nr_p_36_1?ie=UTF8&rs=172659&bbn=172659&rnid=386442011&rh=n%3A1065836%2Cn%3A172659%2Cp%5F4%3ASamsung%2Cp%5F36%3A50000-99999

    Samsung
    TVs & HDTVs
    $500-$999
    Sort by: bestselling

    I want to extract all 48 results from above pages to 1 Excel worksheet

    the data I want in excel file

    ASIN
    Title
    Price
    Manufacturer

    Is it possible to do that? Could you guys give me some advise? Thank you!
     
    siwarin.e, Aug 8, 2008 IP
  2. rohan_shenoy

    rohan_shenoy Active Member

    Messages:
    441
    Likes Received:
    20
    Best Answers:
    0
    Trophy Points:
    60
    #2
    Did you have look at their API?
     
    rohan_shenoy, Aug 8, 2008 IP
  3. Spawny

    Spawny Well-Known Member

    Messages:
    252
    Likes Received:
    23
    Best Answers:
    0
    Trophy Points:
    110
    #3
    you can use their API
     
    Spawny, Aug 8, 2008 IP
  4. AnnyDean

    AnnyDean Peon

    Messages:
    4
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Try with outwit (http://www.outwit.com), seems to work directly with the automatic data structure recognition ('data' tab, 'guess'). It's a FF3 extension that was released recently. Excellent 4 data and image extraction.

    Hope this helps.
     
    AnnyDean, Aug 9, 2008 IP
  5. Edynas

    Edynas Peon

    Messages:
    796
    Likes Received:
    24
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Do it by hand :)
    48 results is easy to handle and before you can digg into the api or work with cURL you will have that sheet ready doing the manual way. But if you need to do this daily or on a larger scale I would also go the api way.
     
    Edynas, Aug 9, 2008 IP
  6. kmap

    kmap Well-Known Member

    Messages:
    2,215
    Likes Received:
    29
    Best Answers:
    2
    Trophy Points:
    135
    #6
    there is a window based package program which can extract info about any book on amazon

    it is easy to make such applications in php or visual basic which can do this

    Hire a programmer and donot ask others to teach you

    Just search on G you will get enough help

    Regards

    Alex
     
    kmap, Aug 9, 2008 IP
  7. AnnyDean

    AnnyDean Peon

    Messages:
    4
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #7
    There are actually 45 results, not 48. OutWit extracted that automatically into an excel file in a snap.

    The Excel file generated includes the list of products. If you want all the detail (Product Dimensions,Shipping Weight,Shipping...) extracted from each product page, you can create a little scraper and have outwit create a more complete Excel file with the data. Didn't do it but that' easy.
     
    AnnyDean, Aug 10, 2008 IP
  8. AnnyDean

    AnnyDean Peon

    Messages:
    4
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #8
    OK, just for the hell of it, I finally made a little scraper to test.
    Can't put too much text in a post, but you can paste these few first lines in Excel to see a fast result:
    ---

    id Product Brand List Price Total Sale Details
    1 Samsung LN32A450 32-Inch 720p LCD HDTV Samsung $885.02 $698.92 720p HD Resolution;Wide Color Enhancer;3 HDMI Inputs;6ms response time
    2 Samsung LN32A330 32-Inch 720p LCD HDTV Samsung $673.52 720p HD Resolution;8,000:1 Contrast Ratio;2 HDMI Inputs;6 ms response time;PC Input
    3 Samsung LN40A550 40-Inch 1080p LCD HDTV Samsung $1,143.05 Full HD 1080p Resolution;Wide Color Enhancer;3 HDMI Inputs;USB 2.0;HDMI-CDC
    4 Samsung LN37A550 37-Inch 1080p LCD HDTV Samsung $1,235.02 $1,037.62 Full HD 1080p Resolution;Wide Color Enhancer;3 HDMI Inputs;USB 2.0;HDMI-CDC
    5 Samsung LN26A450 26-Inch 720p LCD HDTV Samsung $735.02 $609.55 720p HD Resolution;Wide Color Enhancer;3 HDMI Inputs;8 ms response time
    6 Samsung LN37A450 37-Inch 720p LCD HDTV Samsung $851.24 720p HD Resolution;Wide Color Enhancer;3 HDMI Inputs;6ms response time
    7 Samsung LN32A550 32-Inch 1080p LCD HDTV Samsung $901.24 1080p Resolution;15,000:1 Dynamic Contrast Ratio w/ Wide Color Enhancer;5ms response time;3 HDMI, 2 Component, and 1 PC inputs;Swivel Stand
    8 Samsung PN42A450 42-Inch 720p Plasma HDTV Samsung $2,398.99 $2,212.96 720p HD Resolution;20,000:1 Contrast Ratio;3 HDMI Inputs;Filter Bright Anti Reflective Technology;3D Ready for 3D gaming

    (some tabs may be lost in the cut and paste, but it'll give you an idea.)
    ---
    Anny
     
    AnnyDean, Aug 10, 2008 IP
  9. AnonymousUser

    AnonymousUser Peon

    Messages:
    593
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    0
    #9
    curl/libcurl should work for you nicely
     
    AnonymousUser, Aug 10, 2008 IP
  10. siwarin.e

    siwarin.e Peon

    Messages:
    2
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #10
    1. of course, I tried searching/doing it by myself first!
    2. I'm not asking for free help, just a guideline
    3. yes, if it's too complicated for me. I will/want to hire someone to handle this.

    If anyone can do this, PM me.

    Anyway, thank you for your help.
     
    siwarin.e, Aug 10, 2008 IP