Updated Search System

[RadioRob]

Search is one of the most commonly used features within the site.  Given that we have over 125,000 topics and 1.7 million replies, there is a lot of valuable information to be found.  

The default search system used by both Xenforo and Invision is to just directly query the database used for the site.  This system worked, but can be slow and can sometimes miss results.  In IPB, it also did not play nice with full URLs such as links to RM profile pages.  

I've switched the site to use a new search backend known as "ElasticSearch".  This system should return results near instantly and use much less system resources.  

This change is simply on the backend.  So you will continue searching the way you do today.  What results and how they're returned by the system is what is changing.  I'm making this post so no one is surprised if the search results look a little different than they do today.  

To help you get the most out of finding content, below are a few tips that will help get the best results.

RadioRob's Tips for Searching

• When you enter multiple words, the system is applying an "OR" operator to it.  For example.... if you search for:

  apples bananas

  
  The search results would return results which include either apples OR bananas 
• If you want to find an exact phrase, put it in quotation marks.  If you're looking for

  Matt Gaetz

  
  The system will try to prioritize search results that have BOTH phrases, but still return results that have either Matt OR Gaetz in the results.  If you want only the full name, type in:

  "Matt Gaetz" 

  
  to find that exact phrase.  
• Try not to search URLs.  While the system will support it, the results may not be as accurate as they might possibly be.  For example: 

  https://rent.men/bradleyblack

  
  If you search this, the parser is looking at it as:

  http OR rent.men OR bradleyblack

• There are many results that would have two more more of those phrases.  So you can search for 

  "https://rent.men/bradleyblack"

  and get results that have that EXACT result.  

• While this will return the results that match that exact phrase, but you might miss out on some posts about Bradley because there are times when someone only mentions their profile name without the address or they use a different URL such as https://rentmen.eu/bradleyblack.  My recommendation to see the MOST results is to search ONLY the person's username.  

  bradleyblack

Even if you don't decide to take my advice and just search for the full address, it won't just choke up and time out like the old system did before.  It will return results....  but maybe not the best possible results.  

FYI... the new search index is about 75% complete.  It should be finished by 2:00 AM Eastern.  You can actively search now, but just realize some results may not show up yet until it gets finished with the task.  

Happy Searching!

[mike carey]

@RadioRobnot a big deal, more a curiosity, your post above appeared in my unread list yesterday (I know it's the same one not a delete and repost because I had liked it!), but showed up again when I logged on today. Earlier I had seen unread posts that were older, and I knew I had cleared my unread list for the time at which they were dated, but wasn't sure if I had seen them before or not. I guessed at the time that it was a reflection of the new search engine reindexing the data base, or perhaps picking up items that the old software had somehow missed (I was going out on a limb and assuming that the unread posts function uses the search engine to find that content). Even if you choose not to investigate it or even comment (either of which I would understand and accept) it will be interesting to see if it recurs and if so how often.

[RadioRob]

24 minutes ago, mike carey said:

@RadioRobnot a big deal, more a curiosity, your post above appeared in my unread list yesterday (I know it's the same one not a delete and repost because I had liked it!), but showed up again when I logged on today. Earlier I had seen unread posts that were older, and I knew I had cleared my unread list for the time at which they were dated, but wasn't sure if I had seen them before or not. I guessed at the time that it was a reflection of the new search engine reindexing the data base, or perhaps picking up items that the old software had somehow missed (I was going out on a limb and assuming that the unread posts function uses the search engine to find that content). Even if you choose not to investigate it or even comment (either of which I would understand and accept) it will be interesting to see if it recurs and if so how often.

There were a few index rebuilds that ran yesterday.  The first one failed and I had to start it over, meaning at some point yesterday... there were 3 indexes.  (The original MySQL index, the failed ElasticSearch index, and the valid ElasticSearch index).  It's possible when you marked it as read you caught it while the final rebuild was happening.  Unfortunately I already cleaned up the logs from the failed index build, so I can't go parse through to see if that happened.  (The main application had to be reinstalled with a slightly older version than what is available as there is a bug in the newest version that makes it not work with our site.  So it was an uninstall of the newest version, deleting the index it created, installing the older version, and then running it again.)

I've tested marking the site read a few times and have logged out/in each time to confirm the posts are no longer showing as unread so it looks like (at least for me) that it is not still happening on a regular basis.  If you see it again, let me know.  

[+ LS299B]

Great job RadioRob! Thanks for your tenacity and commitment to the rebuild.

Edited June 29, 2021 by LS299B
misspelled RadioRob :-)

[+ MysticMenace]

On 6/28/2021 at 12:50 AM, RadioRob said:

Search is one of the most commonly used features within the site.  Given that we have over 125,000 topics and 1.7 million replies, there is a lot of valuable information to be found.  

The default search system used by both Xenforo and Invision is to just directly query the database used for the site.  This system worked, but can be slow and can sometimes miss results.  In IPB, it also did not play nice with full URLs such as links to RM profile pages.  

I've switched the site to use a new search backend known as "ElasticSearch".  This system should return results near instantly and use much less system resources.  

This change is simply on the backend.  So you will continue searching the way you do today.  What results and how they're returned by the system is what is changing.  I'm making this post so no one is surprised if the search results look a little different than they do today.  

To help you get the most out of finding content, below are a few tips that will help get the best results.

RadioRob's Tips for Searching

• When you enter multiple words, the system is applying an "OR" operator to it.  For example.... if you search for:

  apples bananas

  
  The search results would return results which include either apples OR bananas 
• If you want to find an exact phrase, put it in quotation marks.  If you're looking for

  Matt Gaetz

  
  The system will try to prioritize search results that have BOTH phrases, but still return results that have either Matt OR Gaetz in the results.  If you want only the full name, type in:

  "Matt Gaetz" 

  
  to find that exact phrase.  
• Try not to search URLs.  While the system will support it, the results may not be as accurate as they might possibly be.  For example: 

  https://rent.men/bradleyblack

  
  If you search this, the parser is looking at it as:

  http OR rent.men OR bradleyblack

• There are many results that would have two more more of those phrases.  So you can search for 

  "https://rent.men/bradleyblack"

  and get results that have that EXACT result.  

• While this will return the results that match that exact phrase, but you might miss out on some posts about Bradley because there are times when someone only mentions their profile name without the address or they use a different URL such as https://rentmen.eu/bradleyblack.  My recommendation to see the MOST results is to search ONLY the person's username.  

  bradleyblack

Even if you don't decide to take my advice and just search for the full address, it won't just choke up and time out like the old system did before.  It will return results....  but maybe not the best possible results.  

FYI... the new search index is about 75% complete.  It should be finished by 2:00 AM Eastern.  You can actively search now, but just realize some results may not show up yet until it gets finished with the task.  

Happy Searching!

Aww, and it looks like Bradley Black no longer has his profile up - from what I remember he was hot AF. Sad, coz I never got to meet him.

[RadioRob]

30 minutes ago, JoeMendoza said:

Aww, and it looks like Bradley Black no longer has his profile up - from what I remember he was hot AF. Sad, coz I never got to meet him.

The community consensus is he’s now:

https://rent.men/KyleKinggg

[+ EVdude]

Thanks for the Search Tips. After copy pasting your Matt Gaetz example, I’ve figured out the problem I’m having. In your ‘exact phrase’ example:

"Matt Gaetz" 269 results found

Typing on my phone:
“Matt Gaetz” 5,955 results found

The quotation marks on my iPhone are not the same as in your example.

How does one get the quotation marks in your example? 

Possible to make the system accept the default marks on the iPhone?

thanx!! 

[RadioRob]

If you’re having problems with your phone automatically converting quotation marks, don’t use them. Just search Matt Gaetz. When you see the bazillion results you’ll see an option to refine the results.  It should look something like:


If you click The phase “Matt gaetz”, it will change your results to have the correct quotation marks without you needing to do anything on to your phone to change how quotes are done.