I've replaced the vBulletin 4.x search (which was very slow and caused table locking issues) with a custom one I wrote based on Sphinx. The net effect is that searches should be more relevant and infinitely faster. Fast enough that it's able to spit out query results that would be insane to try to do with SQL (for example on member profiles, you can see what areas they post in the most). Anyway... long story short... search should be much, much, MUCH faster now.
Very nice. I'm a big fan of the What's New / New Posts button, and that took sometimes ages to load. Seems to be very fast now. Love the Post Area section on the profiles, very interesting. So your new search engine still fetches data from the MySQL Server, right? Or does it fetch the data from a cached/dumped directory?
The system does a full reindex of everything in the forum weekly (takes about 10 minutes to do that), then internally it's keeping track of anything that's new, changed or deleted since the last full index, then keeps a much smaller delta index of the changes. The delta index is rebuilt every 5 minutes and takes about 1 second to rebuild. So when running queries, you aren't directly hitting the MySQL servers any longer. Also as a side note, the index rebuilding is transparent and people can still run queries properly when it's rebuilding. Because of the way it does the delta indexing new stuff won't show up for search for up to 5 minutes (until the recurring 5 minute process runs).
Nice idea and I'm sure the 5 minute delay won't hurt anyone. 10 minutes to reindex this massive forum? And only 1 second to rebuild your delta index! Jeez May I ask is your Forum here running on several cluster servers, or just one monster machine? And do you have the SQL Database on a separate machine?
The display of search results has been improved when viewing results as posts (like when you search for all posts by a particular user). Now you can see a preview of the post within the search results.
It's a known bug logged with vBulletin and hopefully they fix it soon. If not, I'll go in and fix it myself...
Search returns no results for me for the same search parameters that usually returned hundreds of posts before this new search engine feature. Why is that?
The problem looks to be in the "(no prefix)" parameter. When I try to search for anything (Posts, Entire post, All Open Forums) and mark "(no prefix)" parameter I don't get any results. Even if I mark other prefixes along with "(no prefix)" I get nothing found, but when I remove "(no prefix)" I get the results for other prefixes.
I have always found the search feature on all forums to be really bad. They do not return the intended keyword you are looking for. It would be cool if it had google like features or even more accruate searching it would make life a lot easier.
Yeah most Forums have a very poor accuracy within their own built in search engine. I often use google for it: Site:forums.digitalpoint.com "my keywords" I wonder if shawns own search engine is more accurate, or if he used the same search algorithm that vBulletin uses.
I'm not sure if this is a bug with vBulletin or the new search engine but when you use the "Containing Text" search on http://forums.digitalpoint.com/entry.php?b=24, the results are just a blank page.
Looks like it's a bug in vBulletin itself... http://www.vbulletin.com/forum/project.php?issueid=33325