Statistics indicate that about every second online store encounters problems with recognizing search queries. This means that users of mobile applications may leave without making a purchase simply because the search couldn't understand their request.
This article explores how to teach the search of a mobile app to recognize the two most common types of requests — exact and erroneous.
Exact Request
When a user knows exactly what they are searching for, they input an exact request, such as "Lay’s chips". This type of request is the easiest to handle technically: full-text search will help.
Full-text search indexes all words and phrases appearing within the app. The index contains information on where these words and phrases occur and determines their weight or relevance.
When a user types "Lay’s chips" into the search, it breaks down the query into "chips" and "Lay’s", searches for these words in the indexes, and finds all documents where they appear. For example, a product card "Lay’s chips with sour cream and herbs", a news article "Try Lay’s chips with the flavor of lightly salted cucumbers", and an article "5 recipes with Lay’s chips".
The system ranks the documents and displays the most relevant to the user. The product card "Lay’s chips with sour cream and herbs" would be ranked highest as it contains all the words from the request, and they are located in the product’s title and description.
Full-text search can be configured using various tools. We most often use Elasticsearch and Diginetica. Diginetica is a service for searching in online stores. It processes all requests automatically. Elasticsearch we configure manually.
Erroneous Request or Typo
In fact, a request with an error or typo is also a type of exact request. The user inputs the product type, its name, or brand incorrectly. For example, "Lays chips" or "chisps Lay’s". Baymard Institute states that 42% of online stores do not recognize such requests, thus risking losing customers who might think the product is out of stock.
Teaching the search to understand erroneous requests can be done in various ways. For instance, integrating libraries or auto-correction modules into the system, creating a custom dictionary of common mistakes and typos, or using phonetic search.
Auto-Correction Libraries and Modules
Elasticsearch includes a built-in function, Elasticsearch Suggester that helps recognize incorrectly spelled queries, offers correction suggestions, and checks spelling. Suggester is easily configured and scales with Elasticsearch.
Hunspell is an open-source spell-checking library that can be added to Elasticsearch as a tokenizer. Additionally, there are Elasticsearch plugins that also allow the integration of Hunspell.
Peter Norvig's Spelling Corrector is an error-correcting algorithm based on probability theory and edit distance — the number of insertions, deletions, or substitutions required to transform one word into another. To integrate Peter Norvig's Spelling Corrector with Elasticsearch, plugins can also be used.
Custom Dictionary of Errors and Typos
A custom dictionary of frequent errors and typos helps improve the recognition of erroneous requests, increase search accuracy, and enhance user experience. This dictionary can be created in Elasticsearch and enhanced with different functions.
Phonetic Search
Phonetic search can use various algorithms, such as Soundex, Metaphone, Double Metaphone, and NYSIIS. They transform words into codes based on their sounds, aiding in processing erroneous requests.
To configure phonetic search in Elasticsearch:
- Create an Elasticsearch index and add a field for phonetic representation of words.
- Use the Elasticsearch "phonetic" analyzer to convert words into their phonetic representation.
- Index data with phonetic representation of words.
- Use Elasticsearch phonetic search to find "Lay’s chips" with a query like "chips leys".
Besides exact and erroneous requests, there are other types: requests by characteristics, compatibility, or slang. If you have questions or would like to discuss your project, write to us. We will help find the right solution.
