Is it possible to pull data from a third party website and place in a database?

Hi all,

I have exactly zero experience in web design, database, and scripts. I am not asking for detailed answers as I would not know what you were talking about anyway. I have a concept that I would like to try to make happen. I know I have a long, arduous, road ahead of me.

After some thought, I have realized that my entire concept rests on the ability to take data from various web sites and place it into a database. This database would then need to be refreshed every 60 seconds or so. An "automatic updating database" if you will, which after set up, requires no intervention from me. The data being retrieved from web sites would most likely be numerical. The database would be "overwritten", as past data would not be necessary. This database would then be used by my website.

If you could simply tell me whether or not this is possible, that would be great. Maybe throw in a few places to get me started learning.

Thanks for taking the time to respond.

-Slim

 

slim ,

sure you will have to create a script/program in java, php, perl, asp, etc to accomplish this task
this commonly known as a web scrapper

 

Hmm... My understanding of a scraper deals with rss feeds. I will be trying to retrieve specific data from a web page (e.g. an opinion poll percentage number). Then drop that number into my database. Does your "yes" answer still apply?

 

Yes it is possible using curl but if you want to pull data from the database you should have a permission to do so

 

I have no intention of shady doings. I Am working under the assumption that I will not have access to other's databases. Is access to their database a requirement?

 

You can scrap the websites and get their data unto your database

 

If it's allowed in your php.ini you can use file_get_contents to make things easy.
Basically:
$scrape=file_get_contents("http://www.site.com/page.php");
preg_match("/regexhere/",$scrape,$found);
insert $found[1] into database.


I know, you don't understand that code. I'm just illustrating that it's really easy. I've done it quite a few times. pm me your idea and I'll help if I can.

 

Great! Thanks for the input. I will certainly pm you in the next couple days. In the meantime could you recommend a few places for a beginner to begin learning code? I hear google has a "playground" to learn. I plan on starting there.

 

w3schools.com - best place to get your feet wet on differnt things

Its php programming you are after - then you can download Xampp which installs apache, MySQL etc on your machine.

Have fun

 

Hey Thanks!

The w3schools was just what I was looking for to get started!