In my workplace, we have an intranet website developed internally that we use for CRM. The data of our clients are shown there (MS SQL server is used as a backend to store data). The site has a limited search functionality at the moment (it'll try to match the text you type in one of the 3 fields that are indexed -company, staff contact, and director name). I'm trying to find info about what would be the easiest method to implement a more sophisticated search (ideally that makes use of AI), like the one implemented by Google or Amazon on their website. It doesn't have to be so sophisticated, anything better than what I described above would do it. Ideally, the search should be able to guess what you're looking for based on your role, your portfolio of clients, and previous searches. It would be great if it could understand what you mean (i.e search query 'song la la la' that returns as result 'ATC -All around the world' on youtube. It would be great to implement vocal search as well. But again it doesn't have to be that complex. Any info on where to look or which resources are readily available would be greatly appreciated. Do you know if there are (and what would be the best) search engines/AI libraries/API that are open source and that would fit this purpose? If this is not the right place to ask, on which other website or forum do you think should I ask for? Thank you in advance
1st of all, let me tell you that a lot of these AI algorithms are suited EXACTLY for the current platform. For example, I guarantee you that if you somehow get Google Search exact algorithm for your project, the algorithm will behave extremely dumb. Because Google Search algorithm works best with billions and billions of possible results. It has also been improved by millions of input crowd sourced input Data. Secondly, have it in mind that a surprising amount of AI algorithms is common sense and not maths/magic/rocket science. For example: Here's how to implement a basic version of predictive search: Simply record every simple search query Group all related search query together. Rank each query according to how often users search for that exact query. When user starts typing, search the DB for queries containing what the user as already typed and sort them in order of frequency. Looks very simple right? You would be surprised how smart this simple algorithm would be when you implement it and people use the search field regularly. For the other parts of ranking search results, you can't really get a perfect 1 is ALL solution to it. Because you really need to observe how exactly your users are using the search field, observe what they expect etc. Another Very simple algorithm for ranking search results is this: Record every single search query Perform the search with your current algorithm Record the actual search result your user is interested in. Now each time a user clicks on a particular result for a particular search term, the rank for that result for that particular search term should be incremented. Now in your future search results for a search term, order it by that rank. The above process looks very basic right? But it behaves very intelligently when used with a lot of users. In this case, you can consider each time a user searches as the training time. Google actually implements these 2 in it's search. Obviously it has a lot of other very sophisticated algorithms to rank web pages. For example: Google knows users prefer pages with https to pages with http They know if a user leaves a web page early, the page is of low quality They know if a web page takes over 4 secs to load, users might not like it With a lot of data, Google has observed that pages with a lot of links linking to (Back links) it are preferred by users. It has observed that if a page contains a range of density for a particular keyword, then that page most likely talks about that keyword. These are things I observed from personal research and experiments. As you can see, most of these things will not apply to your use case. So in this case, I advice you forget Google, Amazon, Playstore search cause they have little in common. Concentrate more on how your own users are using your search. And think up ways to rank results better.