How does big local directories get their data?

Discussion in 'Directories' started by jingCo, Nov 23, 2010.

  1. #1
    How do big local directories get their listing data? The local Canadian one weblocal.ca has 13,000,000 pages indexed. Is it automatic/do they send spiders?

    How about yelp? Is it all user generated?
     
    jingCo, Nov 23, 2010 IP
  2. noahqw

    noahqw Member

    Messages:
    180
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    28
    #2
    Yes, if they have over 13,000,000 pages indexed they're using web spiders to retrieve that massive amount of data.
     
    noahqw, Nov 23, 2010 IP
  3. newlogo

    newlogo Peon

    Messages:
    3,931
    Likes Received:
    11
    Best Answers:
    0
    Trophy Points:
    0
    #3
    every one take data from parent dir and big search engines like google, yahoo, bing
     
    newlogo, Nov 25, 2010 IP
  4. jingCo

    jingCo Well-Known Member

    Messages:
    136
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    120
    #4
    Doesn't those make it illegal to take their content? Do they have another 3rd party provider of content?
     
    jingCo, Nov 29, 2010 IP
  5. fastreplies

    fastreplies Banned

    Messages:
    2,336
    Likes Received:
    107
    Best Answers:
    0
    Trophy Points:
    190
    #5
    They are using scraper script.
    Google for it and you'll find a whole bunch of them.

    Some companies will sell you ready to go Country, State, City etc. database.

    :)

    fastreplies
     
    fastreplies, Nov 29, 2010 IP
  6. stephan2307

    stephan2307 Well-Known Member

    Messages:
    1,277
    Likes Received:
    33
    Best Answers:
    7
    Trophy Points:
    150
    #6
    They definately use scrapers or pay someone to provide the data (who is then using a scraper for them)
     
    stephan2307, Nov 30, 2010 IP
  7. jingCo

    jingCo Well-Known Member

    Messages:
    136
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    120
    #7
    I read in their terms that they pay 3rd party? Who are these 3rd parties? Do they scrap or get it other way?
     
    jingCo, Dec 5, 2010 IP