Can this technically be done ?

hello php pro's, i simply know some basic php not much more than using variables loops arrays etc.
i have not yet worked with mysql other than reading bits of it.
now i have an idea of something which i would like to create and would like your opinion on it
if it can be done the way i would like it...

lets say there is a website where in the content are dates, times and events (format would be for example 25.05.2010 8pm).
now i would like to have an script which will crawl this particular site once a day and if today is listed on
this site (as an event) it should save those times in a database and crawl 5 other sites to find url's.

if first website did have an event at 25.05.10 8pm then those other 5 sites should be crawled every 5 minutes from 7:55pm to 10pm to find and save links that those sites hold and again save them in
the database to be able to show those later on my website.

i hope i am being clear enough, 1 site should be checked for an event only once a day,
if succes (there is a event at a given time) it should start crawling 5 more sites and give me
the links that are stated there. i do not need any content of those sites it should only grasp some links if there are any.

can this be done or am i just full of it ?

if it can be done and one of you guys would like to do this id be happy to recieve some offers also,
but first i would like to know if this kind auf automatization is possible at all

thank you

 

Yeah this should be possible to do

 

do you mean cron jobs?

 

i am not sure if it should be cronjobs, i dont think so but than on the other hand i am not to familiar with it.

the thing is this script should only start scraping if a result is found, not all the time.

any other opinions ?

 

If you want it to run once a day, a cron job is exactly what you need. You can wrap the scraping code in an if block to only run if there is a result.

 

Yes it can be done.

Store the date/timestamp in Database and run the cron jobs every minute. If that date/time is the current time then go and crawl the ones that are in que otherwise just skip and it runs again to check after 1 minute.

24 hours cron job will not work as the time would have already be gone by the time that script is executed tomorrow.