URL scraping

Discussion in 'Programming' started by builtSEO, Jan 14, 2012.

0
  1. #1
    How to do url scraper ?
    Any guidelines ? Any resource website ?

    I knew Php, Mysql. can I write a script to that ?
     
    builtSEO, Jan 14, 2012 IP
  2. Wasi88

    Wasi88 Active Member

    Messages:
    56
    Likes Received:
    4
    Best Answers:
    2
    Trophy Points:
    55
    #2
    If you are just searching for a php framework for url calls then check curl out: http://php.net/manual/en/book.curl.php. If you are searching for a web proxy source code: Use any search engine with something like 'open source php web proxy', 'php web proxy script' ...
     
    Wasi88, Jan 14, 2012 IP
  3. stephan2307

    stephan2307 Well-Known Member

    Messages:
    1,277
    Likes Received:
    33
    Best Answers:
    7
    Trophy Points:
    150
    #3
    You have to use curl. It offers you the options to load urls, send a custom user agent, set referer, accept cookies, upload files and loads more.

    If you need any help let me know as I have loads of experience in scraping.
     
    stephan2307, Jan 17, 2012 IP
  4. robestrong

    robestrong Peon

    Messages:
    33
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Ubot should work too.
     
    robestrong, Jan 20, 2012 IP
  5. www.tetuku.com

    www.tetuku.com Peon

    Messages:
    10
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #5
    or using sockopen. it's more simple
     
    www.tetuku.com, Feb 14, 2012 IP
  6. manju_chandra

    manju_chandra Peon

    Messages:
    167
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #6
    You access to use curl. It offers you the options to bulk urls, advanced a custom user agent, set referer, access cookies, upload files and amaranthine more.
    If you allegation any admonition let me apperceive as I access amaranthine of associate in scraping.
     
    manju_chandra, Feb 14, 2012 IP