Seeking an experienced web crawler/spider developer

Discussion in 'Programming' started by michael scofield, Mar 19, 2009.

0
  1. #1
    Seeking an experienced web crawler/spider developer to devlop a website crawler/spider. The webcrawler/spider will have the following characteristics :

    1. Primary purpose is to crawl a specific number of specific real
    estate sites (i need 3 sites at the moment) for property listings and return the specific listing
    results to a MySQL database.

    2. The crawler will be scheduled to crawl a specified number
    of major real estate sites for listings once a day. (mainly the 3 sites i want)

    3. The crawler will crawl a specified number of minor real estate
    sites for listings once a week.[--optional. Depends on the rate which u ask me.]

    4. Each site being crawled will have differing layouts, and so
    it must be possible to intelligently specify per site
    the layout and what to search for/return.

    5. The crawler must have the ability to compare previous days returned
    results and current results to identify new listings [recently
    added]

    6. The crawler must run on a Linux server and be capable of being
    scheduled to automatically run at certain times.

    7. Admin functions must be allowed to enable the addition/removal of
    sites and to update layout instructions per site.[--optional.]

    Please reply with specific experience and examples of crawlers/spiders you have developed that meet similar requirements.

    PLEASE NOTE : Generic replies with long lists of irrelevant sites will be DELETED from my PM.

    I MAY/MAY NOT OMIT CERTAIN FEATURES TO KEEP IN TUNE WITH MY BUDGET. I Will even consider to make the crawler, crawl just 1 or 2 sites, for money sake. Can you do this for a budget 150-200usd?

    I WILL OMIT MANY FEATURES I DEMAND, IF YOU THINK MY BUDGET OF 150-200 IS TOO LOW, SO THAT THE PROJECT CAN BE DONE WITHIN THIS RANGE ITSELF. AND I WILL ADD MORE FEATURES AS AND WHEN I CAN FUND MORE MONEY FOR THIS, BUT THIS PROJECT CERTAINLY NEEDS TO BE DONE SOON.

    thanks,
     
    michael scofield, Mar 19, 2009 IP
  2. Technovice

    Technovice Peon

    Messages:
    102
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    As Seller:
    100% - 0
    As Buyer:
    100% - 0
    #2
    Please check PM
     
    Technovice, Mar 19, 2009 IP
  3. michael scofield

    michael scofield Guest

    Messages:
    403
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    As Seller:
    100% - 0
    As Buyer:
    100% - 0
    #3
    Please respond interested guys.Really need a programmer.
     
    michael scofield, Apr 15, 2009 IP
  4. harrisunderwork

    harrisunderwork Well-Known Member

    Messages:
    1,005
    Likes Received:
    21
    Best Answers:
    0
    Trophy Points:
    135
    As Seller:
    100% - 0
    As Buyer:
    100% - 0
    #4
    PM Send :)
     
    harrisunderwork, Apr 15, 2009 IP
  5. jom-guru

    jom-guru Banned

    Messages:
    154
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    51
    As Seller:
    100% - 0
    As Buyer:
    100% - 0
    #5
    Well We can do that..............But Your Budget is too Low.......

    We have already done this for Job Sites and Search Engines :) and Porn websites

    Contact me If you are intersted
     
    jom-guru, Apr 16, 2009 IP
  6. yenerich

    yenerich Active Member

    Messages:
    697
    Likes Received:
    7
    Best Answers:
    0
    Trophy Points:
    75
    As Seller:
    100% - 0
    As Buyer:
    100% - 0
    #6
    Drop me a PM.
    I think this job is better for a desktop application. Increase a little your budget and i can do it for you, fast, very optimized and excellent softare.
    If interested send me a PM. Of course explain from which thread are you talking about please since i get many PM talking about similar projects (crawl sites). Thanks
     
    yenerich, Apr 20, 2009 IP