Hello I have a website using google coop but I am tired of it, i d like my own spider, any free script doing that?
Your own spider? And you want to store thousands of sites on your server and plan to have an algorithm which allows you to search through these cached sites within seconds? You need thousands of servers, and unlimited spiders which browse the net continuously and update your database. This is way more complex than you probably image. It's easier to stay with Google, if you ask me.
I have a pretty nice script I can hook you up with. I've built quite a few custom search engines. PM me and maybe we can work something out...
there is no way you are gonna do it better than google combine that search power with yahoo and other major engines and you should show up any results you will ever need
I want to spider only certain sites like google coop does but it s limited to 100 sites , I might have 500, If you know solutions, i am ok
schlogo, There are scripts out there that all you to do what you want without having a ton of server power. I hate to disagree with some of the others, but I do know you can get spidering scripts. I use 3 different types of search engine scripts and spiders, and I only use them on a VPS account. In saying this, the scripts I use are far from being free though. Sorry buddy...
hello I looked at hotscript but nothing really titilating there. aspeek looks like a good solution, It will require some testing but it seems to be ok
Datapark Search Engine http://www.dataparksearch.org/ I've actually used it. Works real well, and fairly customizable. The instructions are not the best, but once you learn how to use it, you can do a lot with it.
All this and not a mention of sphider Quick, simple, easy to install and customize all wrapped up in a LAMP (linux, apache, mysql, php) friendly package. Ooh, yea and it's totally free and open source. Do yourself a favor and check it out. I've used on many occasions, however, this is not meant for a full scale engine like google. But it will work perfectly fine for ~ 500 sites. Google coop sucks because it is all javascript just like their search API. Talk about a paranoid company ....