1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

LF:Software to get all links on a site?

Discussion in 'Link Development' started by wheel, Apr 30, 2006.

  1. #1
    I'm looking for software (free or paid) that will pull all the outbound links from a site. Basically I want to input a url, and end up with a list of all the outbound URLS on any page of the site. I don't care about checking validity, I just want a list.

    Anything out there do this? Lots of link checkers and stuff, but it's never clear if the perform the function that I need.

    TIA.
     
    wheel, Apr 30, 2006 IP
  2. exam

    exam Peon

    Messages:
    2,434
    Likes Received:
    120
    Best Answers:
    0
    Trophy Points:
    0
    #2
    exam, Apr 30, 2006 IP
  3. BILZ

    BILZ Peon

    Messages:
    1,515
    Likes Received:
    62
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Xenu's link sleuth is typically used to help you find borken links, but i think it will also tell you what the outbound links are also. I havent used it in a while. Give it try see if it works: http://home.snafu.de/tilman/xenulink.html
     
    BILZ, Apr 30, 2006 IP
  4. BILZ

    BILZ Peon

    Messages:
    1,515
    Likes Received:
    62
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Hehe, we posted at the same time. Great minds think alike :)
     
    BILZ, Apr 30, 2006 IP
    exam likes this.
  5. wheel

    wheel Peon

    Messages:
    477
    Likes Received:
    19
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Thanks folks!

    If I run this on someone else's site, does it obey robots.txt and behave politely?
     
    wheel, Apr 30, 2006 IP
  6. BILZ

    BILZ Peon

    Messages:
    1,515
    Likes Received:
    62
    Best Answers:
    0
    Trophy Points:
    0
    #6
    not sure. read through the xenu site if you are concerned about it.
     
    BILZ, May 1, 2006 IP
  7. wheel

    wheel Peon

    Messages:
    477
    Likes Received:
    19
    Best Answers:
    0
    Trophy Points:
    0
    #7
    It doesn't obey robots.txt, so I'll check it manually.

    That's nice software, works well. Hard on a site I would think though. Defaults to 30 open connections at once. I turned it down to 1 :)

    Thanks again for the recommendations folks. Looks like exactly what I needed.
     
    wheel, May 1, 2006 IP