1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.
  2. Better Analytics for WordPress Get It Free

Extracting Yahoo backlinks

Discussion in 'Yahoo' started by Weirfire, Feb 4, 2005.

Thread Status:
Not open for further replies.
  1. #1
    Does anyone know how to extract the Yahoo backlinks with PHP? I want to know if this is possible without too much bother?
     
    Weirfire, Feb 4, 2005 IP
  2. SEbasic

    SEbasic Peon

    Messages:
    6,318
    Likes Received:
    318
    Best Answers:
    0
    Trophy Points:
    0
    #2
    It would involve scraping the actual results pages...

    Y! really need to provide an API soon ;)
     
    SEbasic, Feb 4, 2005 IP
  3. joeychgo

    joeychgo Notable Member Premium Member

    Messages:
    3,362
    Likes Received:
    321
    Best Answers:
    0
    Trophy Points:
    255
    #3
    MSN too IMO
     
    joeychgo, Feb 4, 2005 IP
  4. Weirfire

    Weirfire Language Translation Company

    Messages:
    6,967
    Likes Received:
    361
    Best Answers:
    0
    Trophy Points:
    280
    #4
    Would it be difficult to scrape those figures from the results pages? I'm trying to figure out if this is worth it before I start learning.
     
    Weirfire, Feb 4, 2005 IP
  5. SEbasic

    SEbasic Peon

    Messages:
    6,318
    Likes Received:
    318
    Best Answers:
    0
    Trophy Points:
    0
    #5
    I shouldn't think it would be too hard, but it certianly wouldn't be appreciated by Yahoo...
     
    SEbasic, Feb 4, 2005 IP
  6. Weirfire

    Weirfire Language Translation Company

    Messages:
    6,967
    Likes Received:
    361
    Best Answers:
    0
    Trophy Points:
    280
    #6
    I wouldnt be displaying any data that I extract. I would be using it more for calculations ;) Would they still be disappreciative?
     
    Weirfire, Feb 4, 2005 IP
  7. SEbasic

    SEbasic Peon

    Messages:
    6,318
    Likes Received:
    318
    Best Answers:
    0
    Trophy Points:
    0
    #7
    It's the automated checking that would piss them off I think...
     
    SEbasic, Feb 4, 2005 IP
  8. Weirfire

    Weirfire Language Translation Company

    Messages:
    6,967
    Likes Received:
    361
    Best Answers:
    0
    Trophy Points:
    280
    #8
    Should I ask their permission first do you think?
     
    Weirfire, Feb 4, 2005 IP
  9. SEbasic

    SEbasic Peon

    Messages:
    6,318
    Likes Received:
    318
    Best Answers:
    0
    Trophy Points:
    0
    #9
    It depends on how busy you think the tool will be really...

    If Y! start receiving 10000 queries an hour from one IP, I have no doubt that you'll be banned before you can say "Black Hat"... ;)

    I dunno really.
    I'd probabally just do it without asking, but that's me...
     
    SEbasic, Feb 4, 2005 IP
  10. Weirfire

    Weirfire Language Translation Company

    Messages:
    6,967
    Likes Received:
    361
    Best Answers:
    0
    Trophy Points:
    280
    #10
    I reckon if everything works out I could be using it about 20 times a day.

    Surely a site like http://www.marketleap.com/publinkpop/ would be using it a heck of a lot. Do you think they have mutual consent from Yahoo and the other search engines from extracting this data?
     
    Weirfire, Feb 4, 2005 IP
  11. SEbasic

    SEbasic Peon

    Messages:
    6,318
    Likes Received:
    318
    Best Answers:
    0
    Trophy Points:
    0
    #11
    No idea at all mate... Sorry :)

    If you're only doing 20 queries you'd get away with it I'm sure...

    Don't quote me on this though... ;)
     
    SEbasic, Feb 4, 2005 IP
  12. Weirfire

    Weirfire Language Translation Company

    Messages:
    6,967
    Likes Received:
    361
    Best Answers:
    0
    Trophy Points:
    280
    #12
    I have to figure out how to do the queries first though :)

    I guarentee as soon as I work the whole thing out they will bring out an API. I write too much of my own stuff rather than finding scripts already out there. I reckon it's better for me to learn all these languages anyway and get accustomed to writing my own scripts.
     
    Weirfire, Feb 4, 2005 IP
  13. nevetS

    nevetS Evolving Dragon

    Messages:
    2,544
    Likes Received:
    211
    Best Answers:
    0
    Trophy Points:
    135
    #13
    I've read that you need to be under 1 query/second to really piss them off (i.e. ban your address). 20 queries a day is like a normal user. You can use curl to grab the page - and you can grab the # of back links w/in one query.

    Otherwise, if you want to get the sites I'd use LWP and set prefs at 100 results, set the user agent to lynx or something like that and not run more than one query within 5 seconds. The delay will keep you on their good side.

    I've sent a spider at yahoo that pissed them off and temporarily blocked my usage. Something like 950 pages one right after the other. I was just trying out a scraping program I grabbed. I'm sure others have seen that kind of behavior using some of the old yahoo scraping programs out there. Most new programs have anti-bombardment timing built in.
     
    nevetS, Feb 4, 2005 IP
  14. Weirfire

    Weirfire Language Translation Company

    Messages:
    6,967
    Likes Received:
    361
    Best Answers:
    0
    Trophy Points:
    280
    #14
    Thanks nevetS I'll have a look at curl.

    Never used it before. Is it like perl?
     
    Weirfire, Feb 4, 2005 IP
  15. Liminal

    Liminal Peon

    Messages:
    1,279
    Likes Received:
    63
    Best Answers:
    0
    Trophy Points:
    0
    #15
    I'd still shoot them a quick email to describe what you are doing and ask if it's permissible... just to be on the safe side... even if they don't reply for a while, you have a record (sent email) of your question to them
     
    Liminal, Feb 8, 2005 IP
  16. mojtata

    mojtata Peon

    Messages:
    722
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #16
    They won't permit this. And why should you bother
     
    mojtata, Aug 8, 2009 IP
  17. mojtata

    mojtata Peon

    Messages:
    722
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #17
    use them as long before they block you (usually 300 quesries).

    Also there is software called backlinks seo elite google and find it. It extract yahoo backlinks
     
    mojtata, Aug 8, 2009 IP
  18. christyzen

    christyzen Peon

    Messages:
    647
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #18
    Sorry i really dono. But please let me know after you get answer for this thread. Thanks in Advance.
     
    christyzen, Aug 10, 2009 IP
  19. mojtata

    mojtata Peon

    Messages:
    722
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #19
    yahoo gives you csv for download :)
     
    mojtata, Aug 15, 2009 IP
  20. FifthDimension

    FifthDimension Member

    Messages:
    294
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    43
    #20
    I guess Yahoo has some API or something which is what Marketleap is using - so you may want to check out the yahoo BOSS service API whether it has this option available as part of the API.
     
    FifthDimension, Aug 15, 2009 IP
Thread Status:
Not open for further replies.