Need website crawler made.

Discussion in 'Programming' started by domainsurfer, Jan 4, 2008.

0
  1. #1
    I need a website crawler which visits a website, follows all links in that domain and extracts data (email addresses, userids etc.)

    Need a quote for this. PM me with your prices.
    Frontend can be web-based or windows interface. But It should work as desired.
     
    domainsurfer, Jan 4, 2008 IP
  2. undir

    undir Peon

    Messages:
    696
    Likes Received:
    9
    Best Answers:
    0
    Trophy Points:
    0
    As Seller:
    100% - 0
    As Buyer:
    100% - 0
    #2
    I can code this for you for just $180
    Please PM me if interested.
     
    undir, Jan 4, 2008 IP
  3. matthewrobertbell

    matthewrobertbell Peon

    Messages:
    781
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    0
    As Seller:
    100% - 0
    As Buyer:
    100% - 0
    #3
    I can do it, please PM me.
     
    matthewrobertbell, Jan 5, 2008 IP
  4. domainsurfer

    domainsurfer Well-Known Member

    Messages:
    922
    Likes Received:
    20
    Best Answers:
    0
    Trophy Points:
    110
    As Seller:
    100% - 0
    As Buyer:
    100% - 0
    #4
    My budget is 50$
     
    domainsurfer, Jan 16, 2008 IP
  5. DarkMindZ

    DarkMindZ Guest

    Messages:
    175
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    0
    As Seller:
    100% - 0
    As Buyer:
    100% - 0
    #5
    Okay, whats the etc etc..?

    I get that, this bot will go to your site, and extract all links, then visit them.. but then? extarct data to? and can you list all the data you want..

    btw, for a $50, this doesnt look its going to get done. but we will see after you list what you need :)
     
    DarkMindZ, Jan 16, 2008 IP
  6. domainsurfer

    domainsurfer Well-Known Member

    Messages:
    922
    Likes Received:
    20
    Best Answers:
    0
    Trophy Points:
    110
    As Seller:
    100% - 0
    As Buyer:
    100% - 0
    #6
    Just need email addresses. The bot should follow all "do-follow links" ONLY within that domain.
     
    domainsurfer, Jan 16, 2008 IP
  7. DarkMindZ

    DarkMindZ Guest

    Messages:
    175
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    0
    As Seller:
    100% - 0
    As Buyer:
    100% - 0
    #7
    So you want an email crawler?

    If no one takes the job for 2 days, I most prolly will, just gotta get some things done first.
     
    DarkMindZ, Jan 16, 2008 IP
  8. domainsurfer

    domainsurfer Well-Known Member

    Messages:
    922
    Likes Received:
    20
    Best Answers:
    0
    Trophy Points:
    110
    As Seller:
    100% - 0
    As Buyer:
    100% - 0
    #8
    Sure, but do PM me the details, as to how it will work, web based / windows based etc.
     
    domainsurfer, Jan 16, 2008 IP
  9. shallowink

    shallowink Well-Known Member

    Messages:
    1,218
    Likes Received:
    64
    Best Answers:
    2
    Trophy Points:
    150
    As Seller:
    100% - 0
    As Buyer:
    100% - 0
    #9
    I have a perl script setup to spider. PM if you still need this done with an idea of your requirements.
     
    shallowink, Jan 16, 2008 IP
  10. phpl33t

    phpl33t Banned

    Messages:
    456
    Likes Received:
    33
    Best Answers:
    0
    Trophy Points:
    0
    As Seller:
    100% - 0
    As Buyer:
    100% - 0
    #10
    perl? ouch

    dinosaur
     
    phpl33t, Jan 17, 2008 IP
  11. shallowink

    shallowink Well-Known Member

    Messages:
    1,218
    Likes Received:
    64
    Best Answers:
    2
    Trophy Points:
    150
    As Seller:
    100% - 0
    As Buyer:
    100% - 0
    #11
    ah look l33tism, not everybody has a one tool toolbox :)
     
    shallowink, Jan 17, 2008 IP
    vertigoflow likes this.