Is it possible to build "replacement words" into Fuzzy Search?

Brandon Hassler

Fuzzy Search addition has been a game changer to Xano. However, one thing I'm noticing as I test out searches is that there are certain words that mean the same thing but result in low/no matches. 
An ExampleLet's say the user searches "1080p monitor" but I have products that are listed as an "FHD Monitor". Us humans know that 1080p and FHD are the same thing. Or a user may search "gaming prebuild" when many of my products are listed as a "Gaming Desktop PC". Us humans know they both mean the same thing.

Is there a way to associate these words together to improve fuzzy search? 
Potential SolutionThe closest solution I can think of is to create an additional text field for each product and then run some type of conditional function stack like:

IF any fields on [Product A] contain "FHD" THEN add "1080" and "1080p" to the "Additional Words" field. And I would have a function stack that would build overtime with new words & phrases that I notice users are using.

Is there a more efficient way to accomplish this in Xano that I'm not aware of? It would also be nice if Xano allowed us to select the table reference names as part of the Fuzzy Search. I have my products in categories on another table and currently I'm not able to select that as a variable.

Michael Udinski

Hi  - the potential solution you have written out is definitely a good one. I'd stick with that for the time being. I'm sure there will be future expansions of fuzzy search but nothing in the works just yet. 

Ray Deck

 your question touches on a bigger topic in the areas of computational linguistics and machine learning - it's called "word embedding." The idea is that words convey concepts, and the same concept might be described with multiple words, or the intersections of words. 

You can definitely hack it by using a dictionary of things that you know mean the same thing to throw more into the pot. Like seeing "FHD" and adding "1080p" to the mix of words to be searched.

Purpose-built search engines do this. Behind the scenes of Google is a big ol' natural language processing (NLP) engine that takes our misspelled words and figure out the meaning that goes with them. You can use systems like Algolia to get some of this benefit - they have APIs you can hook into Xano like any other.

Seems like the kind of conceptual and mental model thing that is often part of our office hours on the Statechange PRO community! Since you're a member, maybe we'll resume this conversation over there sometime soon.

Brandon Hassler

 Yeah I've had a few friends mention using 3rd party services for this. At the end of the day, no point in reinventing the wheel and getting myself deeper into that "technical debt" you had talked about in the Statechange office hours!

Appreciate the insight!

David Richardson

I am looking at a similar challenge in sorting through a word salad of data I scraped off a site (1,000s of records) with a few text fields for each record.  Some text fields are very short and some are so verbose as to be useless if one had to search for the needle in the haystack.  The text on the Austin YMCA programs ran on to 4.5 pages in a Word document.   

I don't want to expose the text fields in my website.  I just want to be able to "delete" the incidental words (the, of, many, about etc.) to focus on the distinct unique words. If a visitor is searching for an organization inquiring about a hobby, there IS Google.  But Google doesn't offer the context I want to present to a visitor. 

As a means to accomplish this I could see duplicating a table column and then eliminating those pesky useless wordy words to create a more useful descriptive text.  Here is an example:

Text 1:  (note: AAQG is the Austin Area Quilt Guild)  
"THE AAQG promotes education about the art of quiltmaking and the preservation of quilts by means of workshops and programs offered to its members and the public."

The keyword list that would be easier to index and query might look like this"

Text 2: - as an array
[quilts, education, quiltmaking, preservation, workshops, public]

In a spreadsheet it's fairly easy to eliminate specific words from a column of text.  Is there a way to do this with multiple words at a time in Xano?

The fuzzy search may or may not be practical but indexing the word list would probably speed up a search considerably.

The first step I would like to automate is taking each text fields (Text 1:) and eliminating adjectives and adverbs from the text along with punctuation, "a" and "the" etc. and populating a different field in the record with the array "Text 2:"

The second step would be to index the values of the array for a faster search

David Richardson

I found this resource to perhaps sort out the word salad: https://monkeylearn.com/

It's a keyword analyzer.  May do the job.