1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Extract Data from Web Page

Discussion in 'PHP' started by kolucoms6, Sep 5, 2010.

  1. #1
    What I am looking for a PHP applications which will :

    Extract and Display Data , based on 2 search criterias i.e Last Name and County { I may create Table of few last name and Counties in Mysql } , from "Result HTML" based on HTML Tag ( Name, Address and Phone ) in a Tabular Format.

    Repeat the above logic 4 times for Each Page { Page 1, 2 , 3 ,4 } of Resulted HTML.

    Basically , I want to Pull out Data from Different White Pages and Display them in a Tabular Format.

    Possible ??
     
    kolucoms6, Sep 5, 2010 IP
  2. Rainulf

    Rainulf Active Member

    Messages:
    375
    Likes Received:
    12
    Best Answers:
    0
    Trophy Points:
    85
    #2
    Yes, it's possible. :)

    If you're grabbing data from mysql database, it should be something like this:
    PHP:
    1.  
    2. $sql = new mysqli('localhost', 'user', 'pass') or die($sql->error);
    3. $sql->select_db('db') or die ($sql->error);
    4. $result = $sql->query("SELECT * FROM tablename LIMIT 0, to watever");
    5.  
    6. // draw your tabular thingy
    7. echo "<table>";
    8. while($row = $result->fetch_array( )) {
    9.    echo "<tr><td>{$row['watever']}</td></tr>";
    10. }
    11. echo "</table>";
    12.  
    13. $sql->close( );
    14.  
     
    Rainulf, Sep 5, 2010 IP
  3. kolucoms6

    kolucoms6 Active Member

    Messages:
    1,199
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    75
    #3
    Here is what I am looking for :

    I have a Link which displays some data on HTML format :

    http://www.118.com/people-search.mv...john&Location=london&pageSize=50&pageNumber=1

    Data comes in below format :

    <div class="searchResult regular">
    <h2>Bird John</h2>
    <div class="address">
    56 Leathwaite Road<br />
    London<br />
    SW11 6RS
    </div>
    <div class="telephoneNumber">
    020 7228 5576
    </div>
    </div>

    I want my PHP page to execute above URL and Extract/Parse Data from the Result HTML page based on above Tags as
    h2=Name
    address=Address
    telephoneNumber= Phone Number

    and Display them in a Tabular Format.
     
    kolucoms6, Sep 5, 2010 IP
  4. kolucoms6

    kolucoms6 Active Member

    Messages:
    1,199
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    75
    #4
    I got this but it only shows the TEXT format of an HTML page but works to an extent:

     
    kolucoms6, Sep 6, 2010 IP
  5. MyVodaFone

    MyVodaFone Well-Known Member

    Messages:
    1,046
    Likes Received:
    41
    Best Answers:
    10
    Trophy Points:
    195
    #5
    EDIT: See below...
     
    Last edited: Sep 6, 2010
    MyVodaFone, Sep 6, 2010 IP
  6. kolucoms6

    kolucoms6 Active Member

    Messages:
    1,199
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    75
    #6
    Thanks a lot
     
    kolucoms6, Sep 6, 2010 IP
  7. kolucoms6

    kolucoms6 Active Member

    Messages:
    1,199
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    75
    #7
    Tried this to display data in tabular format

    echo '<table border=1><tr><td>'.$name.'</td><td>'.$address.'</td><td>'.$telephoneNumber.'</td></tr></table>';

    but

    format goes heywire.
     
    kolucoms6, Sep 6, 2010 IP
  8. MyVodaFone

    MyVodaFone Well-Known Member

    Messages:
    1,046
    Likes Received:
    41
    Best Answers:
    10
    Trophy Points:
    195
    #8
    Sorry my bad, there's a problem with my code, I'll post back here when I have it, but if anyone else would like to join in, please do... you all know I'm crap at this regex stuff:)
     
    MyVodaFone, Sep 6, 2010 IP
  9. kolucoms6

    kolucoms6 Active Member

    Messages:
    1,199
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    75
    #9
    What I am looking for is :

    Display Option to select Last name and County from the DropDown.

    User can select Atmost 25 Lastnames at a time from each County.

    Once click on Submit, It will execute the Loop of Last name in that particular County and Displays the Result in a Tabular format.

    Is it possible ?
     
    kolucoms6, Sep 6, 2010 IP
  10. MyVodaFone

    MyVodaFone Well-Known Member

    Messages:
    1,046
    Likes Received:
    41
    Best Answers:
    10
    Trophy Points:
    195
    #10
    Ok try this instead and just replace the echo with your table. Note the changes below.

    PHP:
    1.  
    2. <?php
    3.  
    4. $url = get_data("http://www.118.com/people-search.mvc?Supplied=true&Name=john&Location=london&pageSize=50&pageNumber=1");
    5.  
    6. function get_data($url)
    7. {
    8. $ch = curl_init();
    9.     curl_setopt($ch, CURLOPT_HEADER, 0);
    10.     curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    11.     curl_setopt($ch, CURLOPT_URL, $url);
    12.     curl_setopt ($ch, CURLOPT_REFERER, 'http://www.mse360.com/about/bot.php');
    13.     curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
    14.     curl_setopt ($ch, CURLOPT_TIMEOUT, 60);
    15.     $data = curl_exec($ch);
    16.     curl_close($ch);
    17.     return $data;
    18. }
    19.  
    20. $string = preg_match_all('#<h2>([^"]+)</h2>.+?<div class="address">([^"]+)</div>.+?<div class="telephoneNumber">([^"]+)</div>#is', $url, $matches, PREG_SET_ORDER);
    21. foreach ($matches as $item) {
    22. echo '<div class="searchResult regular">
    23.                        <h2>'.$item[1].'</h2>
    24.                        <div class="address">
    25.                        '.$item[2].'
    26.                        </div>
    27.                        <div class="telephoneNumber">
    28.                         '.$item[3].'
    29.                        </div>
    30.                    </div>
    31. ';
    32. }
    33. ?>
    34.  
    With regards to your search criteria, I guess that will be part of your search url

    PS: the address out put, contains <br tags to remove those use strip_tags($item[2])
     
    MyVodaFone, Sep 6, 2010 IP
  11. kolucoms6

    kolucoms6 Active Member

    Messages:
    1,199
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    75
    #11
    Trying this :

    $arr = array(1, 2, 3, 4);
    foreach ($arr as &$value) {
    $url = get_data("http://www.118.com/people-search.mvc?Supplied=true&Name=john&Location=london&pageSize=50&pageNumber=" . $arr);

    But looks like there is an Error as page displays nothing !!!


    Also, echo '<table border=1><tr><td>'.$item[1].'</td><td>'.$item[2].'</td><td>'.$item[3].'</td></tr></table>';
     
    kolucoms6, Sep 6, 2010 IP
  12. MyVodaFone

    MyVodaFone Well-Known Member

    Messages:
    1,046
    Likes Received:
    41
    Best Answers:
    10
    Trophy Points:
    195
    #12

    Whats wrong with what I gave you...
    PHP:
    1.  
    2. <?php
    3.  
    4. $url = get_data("http://www.118.com/people-search.mvc?Supplied=true&Name=john&Location=london&pageSize=50&pageNumber=1");
    5.  
    6. function get_data($url)
    7. {
    8. $ch = curl_init();
    9.     curl_setopt($ch, CURLOPT_HEADER, 0);
    10.     curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); //Set curl to return the data instead of printing it to the browser.
    11.     curl_setopt($ch, CURLOPT_URL, $url);
    12.     curl_setopt ($ch, CURLOPT_REFERER, 'http://www.mse360.com/about/bot.php');
    13.     curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
    14.     curl_setopt ($ch, CURLOPT_TIMEOUT, 60);
    15.     $data = curl_exec($ch);
    16.     curl_close($ch);
    17.     return $data;
    18. }
    19.  
    20. $string = preg_match_all('#<h2>([^"]+)</h2>.+?<div class="address">([^"]+)</div>.+?<div class="telephoneNumber">([^"]+)</div>#is', $url, $matches, PREG_SET_ORDER);
    21. foreach ($matches as $item) {
    22. echo '<table border=1><tr><td>'.$item[1].'</td><td>'.$item[2].'</td><td>'.$item[3].'</td></tr></table>
    23. ';
    24. }
    25. ?>
    26.  
    Your table just needs fixing up.
     
    MyVodaFone, Sep 6, 2010 IP
  13. kolucoms6

    kolucoms6 Active Member

    Messages:
    1,199
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    75
    #13
    Table thing worked perfectly.

    Now, above url is for Page 1, I want to loop through other 3 pages also. I.e 1 - 4 pages.
     
    kolucoms6, Sep 6, 2010 IP
  14. MyVodaFone

    MyVodaFone Well-Known Member

    Messages:
    1,046
    Likes Received:
    41
    Best Answers:
    10
    Trophy Points:
    195
    #14
    You can create your own search form using the following variables:

    PHP:
    1.  
    2. $name= $_POST['name'];
    3. $location=$_POST['location'];
    4. $pageSize=$_POST['pageSize'];
    5. $pageNumber=$_POST['pageNumber'];
    6.  
    Then you build your $url Name=$name&Location=$location&pageSize=$pageSize&pageNumber=$pageNumber
     
    MyVodaFone, Sep 6, 2010 IP
  15. kolucoms6

    kolucoms6 Active Member

    Messages:
    1,199
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    75
    #15
    kolucoms6, Sep 6, 2010 IP
  16. MyVodaFone

    MyVodaFone Well-Known Member

    Messages:
    1,046
    Likes Received:
    41
    Best Answers:
    10
    Trophy Points:
    195
    #16
    As far as I can tell, you can do something like 5,10,15 etc.. to 50, your probable best to set that at a permanent 50 and just use the page numbers
     
    MyVodaFone, Sep 6, 2010 IP
  17. kolucoms6

    kolucoms6 Active Member

    Messages:
    1,199
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    75
    #17
    What the Problem with below code ::


    Code (Text):
    1.  
    2.  
    3. <html>
    4. <body>
    5.  
    6. <link rel=stylesheet type="text/css" href="CSS/default0.css">
    7.  
    8. <form action="<?php echo $_SERVER["PHP_SELF"]; ?>" method="post">
    9.  
    10. Last Name 1 : <input type=text name=name><br>
    11.  
    12. Location : <input type=text name=location><br><br>
    13.  
    14. <input type="submit" value="submit" name="submit">
    15. </form>
    16. </body>
    17. </html>
    18.  
    19. <?php
    20.  
    21. $name= $_POST['name'];
    22. $location=$_POST['location'];
    23.  
    24. ?>
    25.  
    26. <?php
    27.  
    28. $url1 = get_data("http://www.118.com/people-search.mvc?Supplied=true&Name=$name&Location=$london&pageSize=50&pageNumber=1");
    29. $url2 = get_data("http://www.118.com/people-search.mvc?Supplied=true&Name=$name&Location=$london&pageSize=50&pageNumber=2");
    30. $url3 = get_data("http://www.118.com/people-search.mvc?Supplied=true&Name=$name&Location=$london&pageSize=50&pageNumber=3");
    31. $url4 = get_data("http://www.118.com/people-search.mvc?Supplied=true&Name=$name&Location=$london&pageSize=50&pageNumber=4");
    32.  
    33. function get_data($url)
    34. {
    35. $ch = curl_init();
    36.     curl_setopt($ch, CURLOPT_HEADER, 0);
    37.     curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); //Set curl to return the data instead of printing it to the browser.
    38.     curl_setopt($ch, CURLOPT_URL, $url);
    39.     curl_setopt ($ch, CURLOPT_REFERER, 'http://www.mse360.com/about/bot.php');
    40.     curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
    41.     curl_setopt ($ch, CURLOPT_TIMEOUT, 60);
    42.     $data = curl_exec($ch);
    43.     curl_close($ch);
    44.     return $data;
    45. }
    46.  
    47. $string1 = preg_match_all('#<h2>([^"]+)</h2>.+?<div class="address">([^"]+)</div>.+?<div class="telephoneNumber">([^"]+)</div>#is', $url1, $matches, PREG_SET_ORDER);
    48. foreach ($matches as $item) {
    49.  
    50. echo '<table border=1><tr><td>'.$item[1].'</td><td>'.$item[2].'</td><td>'.$item[3].'</td></tr></table>
    51. ';
    52. }
    53.  
    54. $string2= preg_match_all('#<h2>([^"]+)</h2>.+?<div class="address">([^"]+)</div>.+?<div class="telephoneNumber">([^"]+)</div>#is', $url2, $matches, PREG_SET_ORDER);
    55.  
    56. foreach ($matches as $item) {
    57.  
    58. echo '<table border=1><tr><td>'.$item[1].'</td><td>'.$item[2].'</td><td>'.$item[3].'</td></tr></table>
    59. ';
    60. }
    61.  
    62. $string3= preg_match_all('#<h2>([^"]+)</h2>.+?<div class="address">([^"]+)</div>.+?<div class="telephoneNumber">([^"]+)</div>#is', $url3, $matches, PREG_SET_ORDER);
    63.  
    64. foreach ($matches as $item) {
    65.  
    66. echo '<table border=1><tr><td>'.$item[1].'</td><td>'.$item[2].'</td><td>'.$item[3].'</td></tr></table>
    67. ';
    68.  
    69. }
    70. $string4= preg_match_all('#<h2>([^"]+)</h2>.+?<div class="address">([^"]+)</div>.+?<div class="telephoneNumber">([^"]+)</div>#is', $url4, $matches, PREG_SET_ORDER);
    71.  
    72. foreach ($matches as $item) {
    73.  
    74. echo '<table border=1><tr><td>'.$item[1].'</td><td>'.$item[2].'</td><td>'.$item[3].'</td></tr></table>
    75. ';
    76. }
    77.  
    78. ?>
    79.  
    80.  

    When I click on submit, it takes me back the normal page instead of the result.
     
    Last edited: Sep 6, 2010
    kolucoms6, Sep 6, 2010 IP
  18. MyVodaFone

    MyVodaFone Well-Known Member

    Messages:
    1,046
    Likes Received:
    41
    Best Answers:
    10
    Trophy Points:
    195
    #18
    PHP:
    1.  
    2. <html>
    3. <form action="<?php echo $_SERVER["PHP_SELF"]; ?>" method="post">
    4.  
    5. Last Name 1 : <input type=text name=name><br>
    6.  
    7. Location : <input type=text name=location><br><br>
    8.  
    9. <input type="submit" value="submit" name="submit">
    10. </form>
    11. </html>
    12.  
    13. <?php
    14. $name= $_POST['name'];
    15. $location=$_POST['location'];
    16. $url = "http://www.118.com/people-search.mvc?Supplied=true&Name=$name&Location=$location&pageSize=50&pageNumber=1";
    17.  
    18. if(isset($url)){
    19. $url = get_data($url);
    20. }
    21.  
    22. function get_data($url)
    23. {
    24. $ch = curl_init();
    25.     curl_setopt($ch, CURLOPT_HEADER, 0);
    26.     curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); //Set curl to return the data instead of printing it to the browser.
    27.     curl_setopt($ch, CURLOPT_URL, $url);
    28.     curl_setopt ($ch, CURLOPT_REFERER, 'http://www.mse360.com/about/bot.php');
    29.     curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
    30.     curl_setopt ($ch, CURLOPT_TIMEOUT, 60);
    31.     $data = curl_exec($ch);
    32.     curl_close($ch);
    33.     return $data;
    34. }
    35.  
    36. $string = preg_match_all('#<h2>([^"]+)</h2>.+?<div class="address">([^"]+)</div>.+?<div class="telephoneNumber">([^"]+)</div>#is', $url, $matches, PREG_SET_ORDER);
    37. foreach ($matches as $item) {
    38. echo '<table border=1><tr><td>'.$item[1].'</td><td>'.$item[2].'</td><td>'.$item[3].'</td></tr></table>
    39. ';
    40. }
    41. ?>
    If you need anything more complex, try the programming section.
     
    MyVodaFone, Sep 6, 2010 IP
  19. kolucoms6

    kolucoms6 Active Member

    Messages:
    1,199
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    75
    #19
    Thanks for the Code but where did I make the Mistake so that I shld Learn ?
     
    kolucoms6, Sep 6, 2010 IP
  20. kolucoms6

    kolucoms6 Active Member

    Messages:
    1,199
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    75
    #20
    Address has 3 Lines.

    52 Earls Mill Road
    Plymouth
    PL7 2BX

    How to divide it in 3 Different Column ?
     
    kolucoms6, Sep 6, 2010 IP