Nutch Open source, commercial quality. Most other packages won't handle large indexes. Nutch will easily handle 10's of millions of pages. I'm using it on www.mozdex.com.
I'm going to check this out. Thank you very much. Here is a link which works http://lucene.apache.org/nutch/
I am thinking that a search engine that allows PPC gambling advertising would lead to a very good income. Yahoo and Google dont allow gambling ads
Well, I allow gambling ads on my two search engines. That's not the problem . The problem is getting the traffic for the advertisers. Advertisers won't advertise until you've got traffic.
Creating a search engine is easy (atleast the concept of searching). However it is not easy to drive traffic and to break into the search engine market. Google, Yahoo and MSN alone takes most of the visitors. What you need to ask yourself is "why would people come to xyc search engine compared to going to Google"?
someone should make a porn search engine and a gambling search engin and so on, and so on........... Even though i thick there are many.