I need to scrape all urls from live.com aka bing.com I'm using site:site.com to get all the results but there is quite a lot and it's a little tedious to do manually. Do anyone know of a program that will help me gather all urls from one site ?
They have a api http://www.bing.com/developers/?FORM=ZZLH3 and you can also use their feed http://www.bing.com/search?q=Zhift&go=&form=QBRE&format=rss