Lots of Queries: Google Search API or Google Scraping?

Discussion in 'Google API' started by keyaa, Nov 12, 2008.

  1. #1
    Hi,

    I'm currently developing some stuff based on the Google SOAP Search API, which, as most of you know, requires 1 rare, near-extinct API key per 1000 requests / day.

    A friend of mine has another website up-and-running which also queries Google Search Result Pages - but without using the Search API, instead he's just scraping the SERPs with CURL and parses the HTML. His website does ~500-3000 queries to the Google server per day and he says he hasn't experienced any problems.

    So what's the way to go? What do you use, which problems did you face...?
     
    keyaa, Nov 12, 2008 IP
  2. tukarinfobispak

    tukarinfobispak Peon

    Messages:
    702
    Likes Received:
    15
    Best Answers:
    0
    Trophy Points:
    0
    #2
    I use this method on web diagnostic

    I use this method on Ask Mie

    without SOAP, sometime my ip / site temporary banned by Google SERP Service and redirect the result to http://sorry.google.com/sorry

    That's why currently i have several SOAP APIs to support my sites especially web diagnostic
     
    tukarinfobispak, Nov 12, 2008 IP
  3. tukarinfobispak

    tukarinfobispak Peon

    Messages:
    702
    Likes Received:
    15
    Best Answers:
    0
    Trophy Points:
    0
    #3
    how's going?
    i'm interested to know yours
    seem no reply for long long time.
     
    tukarinfobispak, Dec 14, 2008 IP
  4. cannyinfotech

    cannyinfotech Guest

    Messages:
    105
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Google used to track using IP so it is not possible to get always results but once u upload it in server you can do your things it will work perfectly for others
     
    cannyinfotech, Jan 30, 2009 IP
  5. Kankatee

    Kankatee Peon

    Messages:
    9
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Beware, Google often changes it's css and html structure to ward off SERP scrappers. That said, i have a few non-paying apps on the web that rely entirely on Google search results.
     
    Kankatee, Jan 30, 2009 IP
  6. starangel

    starangel Notable Member

    Messages:
    164
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    208
    #6
    Google improves its SERP algorithm everyday so it is hard to get consistent results.
     
    starangel, Feb 5, 2009 IP
  7. jemekite

    jemekite Well-Known Member

    Messages:
    273
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    138
    #7
    I agree with you...
     
    jemekite, Feb 5, 2009 IP
  8. zealus

    zealus Active Member

    Messages:
    70
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    93
    #8
    If you look closely into their html page structure, you will see that certain patterns always remain the same. Even comments.
     
    zealus, Feb 17, 2009 IP
  9. PowerZ

    PowerZ Peon

    Messages:
    74
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #9
    Yeh some remain the same on the pagees. :)
     
    PowerZ, Feb 18, 2009 IP
  10. miccy

    miccy Well-Known Member

    Messages:
    850
    Likes Received:
    9
    Best Answers:
    0
    Trophy Points:
    128
    #10
    Yeah i am using it to my google seach on my japanese av blog
     
    miccy, Feb 23, 2009 IP
  11. robinko

    robinko Peon

    Messages:
    79
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #11
    SERP algorithm of google always change.
     
    robinko, Feb 26, 2009 IP
  12. glucose

    glucose Peon

    Messages:
    20
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #12
    glucose, Feb 8, 2010 IP