Need a webcrawler that will crawl a forums members list

Discussion in 'Programming' started by zchess, Jun 20, 2011.

  1. #1
    I'm trying to find someone on another forum... I tried searching their username in the members list and nothing showed up meaning they were banned. Tried a bunch of different google searches hoping google indexed their profile, but nothing came up since they didn't have any posts on the forum. This forum has over 90,000 members and the the url for a members profile is x.com/member.php?u=93477 and I don't feel like visiting each profile until I find who I'm looking for.

    I figured the best way to find this person would be to use a webcrawler that would crawl each profile until they found the person's username, but I'm not aware of any software that can do this. Is there any software that can do this? I'd be willing to pay if the price is reasonable, or even hire someone to program the software for me. I don't know how much it would cost though, any chance someone could give me an estimate?

    Thanks
     
    zchess, Jun 20, 2011 IP
  2. The Webby

    The Webby Peon

    Messages:
    1,852
    Likes Received:
    30
    Best Answers:
    1
    Trophy Points:
    0
    #2
    Making the program will be easy, just a loop, some if-else, some regular expression, a match function and good to go..
    However, Think about the time it will be taking in crawling all those pages.. The best option for you to have it developed in Java... It should cost you somewhere around $30-$50 (in Java), $20-$25(in PHP), if you only need to find a profile on one forum.
     
    The Webby, Jun 20, 2011 IP