1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Scraping Video Sites Easy/Hard?

Discussion in 'Programming' started by gder01, Jan 24, 2008.

  1. #1
    Guys I will likely pay someone either here (DP) or on scriptlance to build this type of site for me, but first I need to figure out if it is easy or hard (and expensive).

    Basically I want a site that just takes videos from other sites (dosn't host them) and streams them. However, I would like to get videos from maybe 10 different sites.

    T.ube Site 1
    T.ube Site 2
    .....
    T.ube Site 10

    When a user searches for a video in my site, he will get all 10 sites (and be able to filter them out as he so chooses).

    Also he/she should be able to filter results by rating, views, comments, site. This way he/she can get very specific videos.

    What languages would this need, and how difficult would it be for someone to make this script?
     
    gder01, Jan 24, 2008 IP
  2. AstarothSolutions

    AstarothSolutions Peon

    Messages:
    2,680
    Likes Received:
    77
    Best Answers:
    0
    Trophy Points:
    0
    #2
    It can be done in any serverside language. Personally we prefer the .Net framework which is particularly efficient with screen scraping but the masses on here will follow the normal PHP trend.

    The level of difficulty will be very much dependant on the target sites in terms of their layout, functionality and security.
     
    AstarothSolutions, Jan 25, 2008 IP
  3. amnezia

    amnezia Peon

    Messages:
    990
    Likes Received:
    31
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Its dead easy. I can write that in about 5 lines of perl. Why you would want to use .NET i have no idea....
     
    amnezia, Jan 25, 2008 IP
  4. shallowink

    shallowink Well-Known Member

    Messages:
    1,218
    Likes Received:
    64
    Best Answers:
    2
    Trophy Points:
    150
    #4
    Grabbing any single page is trivial. What you want though is a bit more complicated. The price goes up for every site scraped. You don't say where the 'ratings,comments' comes from. If its part of your site, easy enough. guru.com/getacoder.com prices averages ~$70. Most times that is just for output to a flatfile or DB. Some of your requirements sound specific to presenting the data, so it would cost more. PM me if your timeframe is flexible and I will give you a better idea of cost.
     
    shallowink, Jan 25, 2008 IP
    gder01 likes this.
  5. AstarothSolutions

    AstarothSolutions Peon

    Messages:
    2,680
    Likes Received:
    77
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Because it is one line of .Net to scrape rather than 5 of perl and .Net is pre-compiled :confused:
     
    AstarothSolutions, Jan 25, 2008 IP
  6. shallowink

    shallowink Well-Known Member

    Messages:
    1,218
    Likes Received:
    64
    Best Answers:
    2
    Trophy Points:
    150
    #6
    So how does .Net do on parsing? They have built in HTML parsing? Ain't going to convert to the dark side, just curious.
     
    shallowink, Jan 25, 2008 IP
  7. gder01

    gder01 Active Member

    Messages:
    262
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    53
    #7
    can somoen show some examples of .net vs pearl?
    I dont know anything about programming, but I assume one has advantages over the other when it comes to these tube scraping sites
     
    gder01, Jan 25, 2008 IP
  8. SNaRe

    SNaRe Well-Known Member

    Messages:
    1,132
    Likes Received:
    32
    Best Answers:
    0
    Trophy Points:
    165
    #8
    I'm experineced about data scraping and i wrote too many video sites. I can do this with only PHP & MYSQL. They are enough. I can show you some of my works also. Just send me PM about this subject
     
    SNaRe, Feb 4, 2008 IP
  9. AstarothSolutions

    AstarothSolutions Peon

    Messages:
    2,680
    Likes Received:
    77
    Best Answers:
    0
    Trophy Points:
    0
    #9
    But the dark side is where all the kewl people are ;)

    .Net has built in parsing for html which is fairly strong but must admit to not having used it much or seen benchmarking to see how efficient it is.
     
    AstarothSolutions, Feb 4, 2008 IP