Extract Data from Web Page

What I am looking for a PHP applications which will :

Extract and Display Data , based on 2 search criterias i.e Last Name and County { I may create Table of few last name and Counties in Mysql } , from "Result HTML" based on HTML Tag ( Name, Address and Phone ) in a Tabular Format.

Repeat the above logic 4 times for Each Page { Page 1, 2 , 3 ,4 } of Resulted HTML.

Basically , I want to Pull out Data from Different White Pages and Display them in a Tabular Format.

Possible ??

 

Yes, it's possible.

If you're grabbing data from mysql database, it should be something like this:

$sql = new mysqli('localhost', 'user', 'pass') or die($sql->error);
$sql->select_db('db') or die ($sql->error);
$result = $sql->query("SELECT * FROM tablename LIMIT 0, to watever");

// draw your tabular thingy
echo "<table>";
while($row = $result->fetch_array( )) {
   echo "<tr><td>{$row['watever']}</td></tr>";
}
echo "</table>";

$sql->close( );

PHP:

 

Here is what I am looking for :

I have a Link which displays some data on HTML format :

http://www.118.com/people-search.mv...john&Location=london&pageSize=50&pageNumber=1

Data comes in below format :

<div class="searchResult regular">
<h2>Bird John</h2>
<div class="address">
56 Leathwaite Road<br />
London<br />
SW11 6RS
</div>
<div class="telephoneNumber">
020 7228 5576
</div>
</div>

I want my PHP page to execute above URL and Extract/Parse Data from the Result HTML page based on above Tags as
h2=Name
address=Address
telephoneNumber= Phone Number

and Display them in a Tabular Format.

 

I got this but it only shows the TEXT format of an HTML page but works to an extent:

 

EDIT: See below...

 

Thanks a lot

 

Tried this to display data in tabular format

echo '<table border=1><tr><td>'.$name.'</td><td>'.$address.'</td><td>'.$telephoneNumber.'</td></tr></table>';

but

format goes heywire.

 

Sorry my bad, there's a problem with my code, I'll post back here when I have it, but if anyone else would like to join in, please do... you all know I'm crap at this regex stuff

 

What I am looking for is :

Display Option to select Last name and County from the DropDown.

User can select Atmost 25 Lastnames at a time from each County.

Once click on Submit, It will execute the Loop of Last name in that particular County and Displays the Result in a Tabular format.

Is it possible ?

 

Ok try this instead and just replace the echo with your table. Note the changes below.

<?php

$url = get_data("http://www.118.com/people-search.mvc?Supplied=true&Name=john&Location=london&pageSize=50&pageNumber=1");

function get_data($url)
{
$ch = curl_init();
	curl_setopt($ch, CURLOPT_HEADER, 0);
	curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); 
	curl_setopt($ch, CURLOPT_URL, $url);
	curl_setopt ($ch, CURLOPT_REFERER, 'http://www.mse360.com/about/bot.php');
 	curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
	curl_setopt ($ch, CURLOPT_TIMEOUT, 60);
	$data = curl_exec($ch);
	curl_close($ch);
	return $data;
}

$string = preg_match_all('#<h2>([^"]+)</h2>.+?<div class="address">([^"]+)</div>.+?<div class="telephoneNumber">([^"]+)</div>#is', $url, $matches, PREG_SET_ORDER);
foreach ($matches as $item) {
echo '<div class="searchResult regular">
                        <h2>'.$item[1].'</h2>
                        <div class="address">
                        '.$item[2].'
                        </div>
                        <div class="telephoneNumber">
                         '.$item[3].'
                        </div>
                    </div>
';
}
?>

PHP:

With regards to your search criteria, I guess that will be part of your search url

PS: the address out put, contains <br tags to remove those use strip_tags($item[2])

 

Trying this :

$arr = array(1, 2, 3, 4);
foreach ($arr as &$value) {
$url = get_data("http://www.118.com/people-search.mvc?Supplied=true&Name=john&Location=london&pageSize=50&pageNumber=" . $arr);

But looks like there is an Error as page displays nothing !!!


Also, echo '<table border=1><tr><td>'.$item[1].'</td><td>'.$item[2].'</td><td>'.$item[3].'</td></tr></table>';

 

Whats wrong with what I gave you...

<?php

$url = get_data("http://www.118.com/people-search.mvc?Supplied=true&Name=john&Location=london&pageSize=50&pageNumber=1");

function get_data($url)
{
$ch = curl_init();
	curl_setopt($ch, CURLOPT_HEADER, 0);
	curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); //Set curl to return the data instead of printing it to the browser.
	curl_setopt($ch, CURLOPT_URL, $url);
	curl_setopt ($ch, CURLOPT_REFERER, 'http://www.mse360.com/about/bot.php');
 	curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
	curl_setopt ($ch, CURLOPT_TIMEOUT, 60);
	$data = curl_exec($ch);
	curl_close($ch);
	return $data;
}

$string = preg_match_all('#<h2>([^"]+)</h2>.+?<div class="address">([^"]+)</div>.+?<div class="telephoneNumber">([^"]+)</div>#is', $url, $matches, PREG_SET_ORDER);
foreach ($matches as $item) {
echo '<table border=1><tr><td>'.$item[1].'</td><td>'.$item[2].'</td><td>'.$item[3].'</td></tr></table>
';
}
?>

PHP:

Your table just needs fixing up.

 

Table thing worked perfectly.

Now, above url is for Page 1, I want to loop through other 3 pages also. I.e 1 - 4 pages.

 

You can create your own search form using the following variables:

$name= $_POST['name'];
$location=$_POST['location'];
$pageSize=$_POST['pageSize'];
$pageNumber=$_POST['pageNumber'];

PHP:

Then you build your $url Name=$name&Location=$location&pageSize=$pageSize&pageNumber=$pageNumber

 

I have Tried Pagesize variable but whatever I select It always take Maximum 50 per page. May be server setting is like that .

http://www.118.com/people-search.mv...ohn&Location=london&pageSize=200&pageNumber=1

 

As far as I can tell, you can do something like 5,10,15 etc.. to 50, your probable best to set that at a permanent 50 and just use the page numbers

 

What the Problem with below code ::

<html>
<body>

<link rel=stylesheet type="text/css" href="CSS/default0.css">

<form action="<?php echo $_SERVER["PHP_SELF"]; ?>" method="post">

Last Name 1 : <input type=text name=name><br>

Location : <input type=text name=location><br><br>

<input type="submit" value="submit" name="submit">
</form>
</body>
</html>

<?php

$name= $_POST['name'];
$location=$_POST['location'];

?>

<?php

$url1 = get_data("http://www.118.com/people-search.mvc?Supplied=true&Name=$name&Location=$london&pageSize=50&pageNumber=1");
$url2 = get_data("http://www.118.com/people-search.mvc?Supplied=true&Name=$name&Location=$london&pageSize=50&pageNumber=2");
$url3 = get_data("http://www.118.com/people-search.mvc?Supplied=true&Name=$name&Location=$london&pageSize=50&pageNumber=3");
$url4 = get_data("http://www.118.com/people-search.mvc?Supplied=true&Name=$name&Location=$london&pageSize=50&pageNumber=4");

function get_data($url)
{
$ch = curl_init();
    curl_setopt($ch, CURLOPT_HEADER, 0);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); //Set curl to return the data instead of printing it to the browser.
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt ($ch, CURLOPT_REFERER, 'http://www.mse360.com/about/bot.php');
    curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
    curl_setopt ($ch, CURLOPT_TIMEOUT, 60);
    $data = curl_exec($ch);
    curl_close($ch);
    return $data;
}

$string1 = preg_match_all('#<h2>([^"]+)</h2>.+?<div class="address">([^"]+)</div>.+?<div class="telephoneNumber">([^"]+)</div>#is', $url1, $matches, PREG_SET_ORDER);
foreach ($matches as $item) {

echo '<table border=1><tr><td>'.$item[1].'</td><td>'.$item[2].'</td><td>'.$item[3].'</td></tr></table>
';
}

$string2= preg_match_all('#<h2>([^"]+)</h2>.+?<div class="address">([^"]+)</div>.+?<div class="telephoneNumber">([^"]+)</div>#is', $url2, $matches, PREG_SET_ORDER);

foreach ($matches as $item) {

echo '<table border=1><tr><td>'.$item[1].'</td><td>'.$item[2].'</td><td>'.$item[3].'</td></tr></table>
';
}

$string3= preg_match_all('#<h2>([^"]+)</h2>.+?<div class="address">([^"]+)</div>.+?<div class="telephoneNumber">([^"]+)</div>#is', $url3, $matches, PREG_SET_ORDER);

foreach ($matches as $item) {

echo '<table border=1><tr><td>'.$item[1].'</td><td>'.$item[2].'</td><td>'.$item[3].'</td></tr></table>
';

}
$string4= preg_match_all('#<h2>([^"]+)</h2>.+?<div class="address">([^"]+)</div>.+?<div class="telephoneNumber">([^"]+)</div>#is', $url4, $matches, PREG_SET_ORDER);

foreach ($matches as $item) {

echo '<table border=1><tr><td>'.$item[1].'</td><td>'.$item[2].'</td><td>'.$item[3].'</td></tr></table>
';
}

?>

Code (markup):

When I click on submit, it takes me back the normal page instead of the result.

 

<html>
<form action="<?php echo $_SERVER["PHP_SELF"]; ?>" method="post">

Last Name 1 : <input type=text name=name><br>

Location : <input type=text name=location><br><br>

<input type="submit" value="submit" name="submit">
</form>
</html>

<?php
$name= $_POST['name'];
$location=$_POST['location'];
$url = "http://www.118.com/people-search.mvc?Supplied=true&Name=$name&Location=$location&pageSize=50&pageNumber=1";

if(isset($url)){
$url = get_data($url);
}

function get_data($url)
{
$ch = curl_init();
	curl_setopt($ch, CURLOPT_HEADER, 0);
	curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); //Set curl to return the data instead of printing it to the browser.
	curl_setopt($ch, CURLOPT_URL, $url);
	curl_setopt ($ch, CURLOPT_REFERER, 'http://www.mse360.com/about/bot.php');
 	curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
	curl_setopt ($ch, CURLOPT_TIMEOUT, 60);
	$data = curl_exec($ch);
	curl_close($ch);
	return $data;
}

$string = preg_match_all('#<h2>([^"]+)</h2>.+?<div class="address">([^"]+)</div>.+?<div class="telephoneNumber">([^"]+)</div>#is', $url, $matches, PREG_SET_ORDER);
foreach ($matches as $item) {
echo '<table border=1><tr><td>'.$item[1].'</td><td>'.$item[2].'</td><td>'.$item[3].'</td></tr></table>
';
}
?>

PHP:

If you need anything more complex, try the programming section.

 

Thanks for the Code but where did I make the Mistake so that I shld Learn ?

 

Address has 3 Lines.

52 Earls Mill Road
Plymouth
PL7 2BX

How to divide it in 3 Different Column ?