I know that tools like keycompete.com or keyspy.com give you a list of competitors for keywords that you intend to bid on. If we were to do this manually, we would fire up the browser, go to google, search for each keyword, and note down which competitors are advertising. We want to create this functionality programmatically for an inhouse SEM tool we are building - can someone help with how to do this programmatically? We can scrape the results from a google search result page, but since google disapproves of automatic approaches like this - they may ban the IP the request is coming from. Any bright ideas on this ?
Firstly it sounds illegal, secondly as you mention programatically, maybe you should chat with a programmer. I dont like the sound of it though.
It is NOT illegal as there is no law against that - Google does it continuously to all sites on the net as a main feature of their business but don't like it when someone does it to their site - go figure..
My choice of word is bad referring to it as illegal, but i believe it would be something they disapprove of.
Well scraping part is easy.You just need to constantly rotate the proxies.Else IP would be banned. PM me i you need any details.