plzzz tell me the difference web scraping using google API and the regular scraping..

Discussion in 'Google API' started by trample, Mar 1, 2010.

0
  1. #1
    plzzz tell me the difference web scraping using google API and the regular scraping.. which one is more efficient???
     
    trample, Mar 1, 2010 IP
  2. nosf009

    nosf009 Well-Known Member

    Messages:
    146
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    118
    #2
    Although post is old, I can reply with answer for those who don't know difference:

    Scraping:
    - get many results
    - parse manually /w some script
    - in about 10 days or sooner get blocked by google, which is pretty bad if you need that for a website

    Google API
    - get up to 64 results, 8 per page
    - up to 1000 requests per day per IP
    - structured response, easier to code

    Generally, if you're up to doing some useful white hat stuff, API is way to go.
     
    nosf009, Apr 1, 2010 IP
  3. toni.blue

    toni.blue Greenhorn

    Messages:
    29
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    11
    #3
    nosf009: Thx for the reply!

    Have you used the Google API lately? I mean are you sure you can get 64 results?
     
    toni.blue, Apr 1, 2010 IP
  4. nosf009

    nosf009 Well-Known Member

    Messages:
    146
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    118
    #4
    I'm using API on few of my websites, if we're talking about web search api - that's what I suppose this thread was about.
    You can get up to 64, yes, 8 per page.
     
    nosf009, Apr 2, 2010 IP
  5. nokimchen

    nokimchen Guest

    Messages:
    264
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Well, according to my view, If you really want to scrap the web, then avoid API..
    If you want to scrap, say 10k urls within 24 hrs then you cant do it with API, and if you do scrapping regularly then Google might ban your IP

    So, the solution is proxy!
    Use a proxy and start scrapping.. :p

    note: too much of anything is not good! Be within the limit!
     
    nokimchen, Apr 11, 2010 IP
  6. toni.blue

    toni.blue Greenhorn

    Messages:
    29
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    11
    #6
    nosf009: I've looked on Google's API pages and only found the AJAX search API, which is not really good for scrapping:
    http://code.google.com/apis/ajaxsearch/web.html

    Could you give me more details about the API you're using? Or even better maybe send me an example code or even your code if you don't mind?

    Thank you very much!
     
    toni.blue, Apr 12, 2010 IP
  7. webscraper

    webscraper Peon

    Messages:
    6
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #7
    tony.blue, only the AJAX API is available now. Google's proper Search API was discontinued several years ago. I guess they didn't want any competition.

    nokimchen is right, Proxy and Scrape, though Google is pretty sharp with the Proxy IP Addresses.
     
    webscraper, Apr 18, 2010 IP
  8. Inktcartridges

    Inktcartridges Peon

    Messages:
    15
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #8
    I don't understand it. nosf009 says Google API scrapping is white hat, but then why do we have to use proxies?
     
    Inktcartridges, Jun 11, 2010 IP
  9. traffic maker

    traffic maker Peon

    Messages:
    37
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #9
    Hello: Well one differenbce is if you do too mauch scraping with Google you might get banned.
     
    traffic maker, Jun 17, 2010 IP
  10. khalilivfyl695

    khalilivfyl695 Peon

    Messages:
    12
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #10
    l can not understand you very well!
     
    khalilivfyl695, Aug 10, 2010 IP
  11. iMarketingGuru

    iMarketingGuru Well-Known Member

    Messages:
    486
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    160
    #11
    Use proxy ips if you go the regular braking route but I definitely suggest going the cool way of using their API and possibly getting more leeway as to how much API usage you get by making a popular app
     
    iMarketingGuru, Aug 11, 2010 IP