Extracting Yahoo backlinks

Does anyone know how to extract the Yahoo backlinks with PHP? I want to know if this is possible without too much bother?

 

It would involve scraping the actual results pages...

Y! really need to provide an API soon

 

MSN too IMO

 

Would it be difficult to scrape those figures from the results pages? I'm trying to figure out if this is worth it before I start learning.

 

I shouldn't think it would be too hard, but it certianly wouldn't be appreciated by Yahoo...

 

I wouldnt be displaying any data that I extract. I would be using it more for calculations Would they still be disappreciative?

 

It's the automated checking that would piss them off I think...

 

Should I ask their permission first do you think?

 

It depends on how busy you think the tool will be really...

If Y! start receiving 10000 queries an hour from one IP, I have no doubt that you'll be banned before you can say "Black Hat"...

I dunno really.
I'd probabally just do it without asking, but that's me...

 

I reckon if everything works out I could be using it about 20 times a day.

Surely a site like http://www.marketleap.com/publinkpop/ would be using it a heck of a lot. Do you think they have mutual consent from Yahoo and the other search engines from extracting this data?

 

No idea at all mate... Sorry

If you're only doing 20 queries you'd get away with it I'm sure...

Don't quote me on this though...

 

I have to figure out how to do the queries first though

I guarentee as soon as I work the whole thing out they will bring out an API. I write too much of my own stuff rather than finding scripts already out there. I reckon it's better for me to learn all these languages anyway and get accustomed to writing my own scripts.

 

I've read that you need to be under 1 query/second to really piss them off (i.e. ban your address). 20 queries a day is like a normal user. You can use curl to grab the page - and you can grab the # of back links w/in one query.

Otherwise, if you want to get the sites I'd use LWP and set prefs at 100 results, set the user agent to lynx or something like that and not run more than one query within 5 seconds. The delay will keep you on their good side.

I've sent a spider at yahoo that pissed them off and temporarily blocked my usage. Something like 950 pages one right after the other. I was just trying out a scraping program I grabbed. I'm sure others have seen that kind of behavior using some of the old yahoo scraping programs out there. Most new programs have anti-bombardment timing built in.

 

Thanks nevetS I'll have a look at curl.

Never used it before. Is it like perl?

 

I'd still shoot them a quick email to describe what you are doing and ask if it's permissible... just to be on the safe side... even if they don't reply for a while, you have a record (sent email) of your question to them

 

They won't permit this. And why should you bother

 

use them as long before they block you (usually 300 quesries).

Also there is software called backlinks seo elite google and find it. It extract yahoo backlinks

 

Sorry i really dono. But please let me know after you get answer for this thread. Thanks in Advance.

 

yahoo gives you csv for download

 

I guess Yahoo has some API or something which is what Marketleap is using - so you may want to check out the yahoo BOSS service API whether it has this option available as part of the API.