Designing and Developing Next-Generation Free-Text Search

By Karim Jamal, Chad Huff & Valle Hansen

What does the term “search” mean to you? On any given day, we perform numerous searches, many consciously and many without even realizing. We do several web searches every day, either by keying in search terms or by using a voice assistant. When we enter a location into a maps app before starting to drive, we’re actually searching for the best route to wherever we’re going. And don’t forget the mental search for the car keys that some of us do every morning.

Search is everywhere in our lives. Each of these searches is a different type of search. There are different ways to interact with search technologies, different ways that the information will be retrieved, and different ways in which you will use the information you get.

So what type of search do you design for your application or website? There are a number of considerations to take into account: What do your users want to search for? How are they used to searching? What are the most natural ways for users to search? How do you go about developing a search solution? When tackling this, you have to be aware of the cross-disciplinary considerations that go into making such a decision. Taking into account user perspectives, UX patterns, and development considerations will help create a successful outcome.

Note: In this post we’ll look specifically at text searches.

User Perspectives

There is no real user persona or specific demographic for search, because everybody does it. Search is agnostic of gender, age, health, location, education, income, etc. The key in considering user perspectives on search is to understand not only who searches, but also why users search.

Motivations for Search

Users search for a number of reasons. You can probably look back on your morning and come up with a handful of things you’ve already searched for, and all for different reasons. But those searches might have something in common at a high level. Bubbled up, users search to:

• Know: These searches are informational. The user is typically looking for a cut-and-dried, clear answer to a factual question. In these scenarios, one answer or search result will typically suffice.

• Understand: Sometimes users have an aspirational motivation to search on a topic or a theme. They want to learn about something, so there is no clear endpoint or right or wrong answer. Users will often sift through multiple results when conducting these searches.

• Do: More and more, we’re seeing searches for “how to” do something. These are action-oriented searches that help the user complete a task. Typically one result will suffice, but it’s not always the first result the user sees, so some time is spent seeking the best one.

• Find: Often users look for something that will likely precipitate an action, such as finding a website or a location or a bargain. These anticipatory searches are neither clear-cut nor amorphous; a user can spend time looking for the best deal or the best route.

Now that we know why users are searching, we can begin to look at what they expect from a high-quality textual search experience.

Expecting More from Search

As with most emerging tech trends, users are expecting more and more from their interactions and experiences with search. The key differentiators for next-gen search are:

• Relevance: Search results should reflect that the system understands what the user is asking and knows what to suggest.
• Personalization: Searches and search results are personal to the user, or what the search engine knows about the user.
• Contextualization: Searches and other user activities are linked to one another so search engines have richer context for what the user is seeking and why.

Users search for a variety of reasons, but their behaviors and motivations often share  commonalities across geographic, demographic and psychographic divides. The next step to creating a valuable search experience is to identify how the search mechanism should work.

UX Patterns

Which user interface elements or patterns do you leverage when creating a search tool? There are a few guidelines and patterns we use most often. But first, let’s clarify the difference between search and filter. Filtering takes an existing list or visible data and removes items based on criteria. Searching takes a blank slate and adds to it based on criteria. How we provide the criteria for the search (or filter) depends on the user’s need and the complexity of the query.

Attribute-Based Search is used when needing to specify one or a few attributes to achieve a desired set of search results. This most often looks like a text field for capturing text and numbers with a way to select a type or category to search within, e.g., picking a department to search within Amazon.

In Slack, you can narrow your search before you enter or as you are entering text.  

Another common pattern is Completion Suggestion. When a user may not know the exact details to manually type into a text field, or typing the full information might be inefficient, a drop-down-like menu can appear under the active text box with suggestions for completing the “intent” of what the user has started typing in a field.

Suggestions can be categorized in the list. If there can be confidence in a “top hit”, making a completion suggestion can also show filling in the rest of the text query in a visibly different style. A word of caution to take a suggestion to autocomplete: A user may want to hit “enter” or submit the query incomplete to see all results. Be sure to design for that path as well. (Note that completion suggestion and autocomplete are not the same thing.)

Search Forms are often used when several attributes are required to present useful results. The forms you’ve likely used on travel sites are a good example of these forms that allow the user to define specific criteria.

Expero worked with Intersil to create a configurable search form to define criteria to find solutions (parts).

There are other patterns like Search History that could be useful, but we are seeing new patterns emerge, too. As users begin talking with their devices, we see voice or visual input methods for creating queries as well as passive methods like location. As technology advances, our patterns may have less UI or more UI. The goal is to give the user the best user interface for the query they want to write, build, say, or be.

Development Considerations

When it comes time for how search will be developed, a lot of detailed questions start coming up. This is the point where the tire actually meets the road, so the small details need to be clarified; otherwise, chances of veering off the road increase quickly. From personal experience, we can confidently say that search can mean something different depending on whom you’re asking. Unless the expectations are clearly laid out, chances are that the end result will vary from what was actually expected at the outset.

Getting the requirements correct is not only to make sure that you’re building the right thing, which is obviously very important, but also to be able to estimate the effort and resources needed to complete the feature. A free-text search can get very complex very quickly. For example, let’s take a simple query:

Imagine that this is backed by a simple data store that has a table or node for employee information. This would then translate to getting all employee records where the start date was in 2016. In this case, there is very little preprocessing of the query that is needed to understand its meaning.

It’s easy to see how this type of free-text searching can get a lot more complex. Let’s take a look at a more complex query:

There is a lot more going on here. No simple querying of the data store is going to be able to answer this query in a straightforward way. What makes this so difficult? There are loads of context hidden in this search. The user is expecting the search to know information about herself, so the answer will need to weave together results from several tangential sources. Here’s a closer look at some of the preprocessing the search will have to perform just to understand what the user is asking for:

Comparing the simple and complex queries, it becomes obvious that building the latter can be magnitudes more costly than the former. Building the latter when only the former was needed will lead to a lot of wasted effort. At the same time, building the former when the latter was needed will lead to unamused reactions from your directors. Thus it’s important to figure out how “smart” your search needs to be.

Summing Up: Gathering Requirements

So what questions do you raise and consider early on in the process to make sure you are on the right track with your search tool? We have prepared several questions about the user, the behavior, the search query and the search results to aid in getting these answers.

Users:

• Who will be searching?
• What are they likely to search for?
• What is the intent behind their search?

Design:

• How complex are the search queries?
• How do users want to see results?

Implementation:

• What exactly are we searching? Is it going to be one field in one database table or node? Multiple fields? Multiple tables/nodes? Everything?
• Will there be completion suggestions? If so, what do we show in the suggestions?
• Is there any sorting or relevance that needs to be applied to the suggestions?
• Do the suggestions need to be grouped in any way (e.g., “top hit”)?
• How many suggestions should be shown?

Armed with this question checklist and a better understanding of how the different disciplines influence your search experience, you will be better equipped to build the appropriate experience for your users. Additionally, a clear understanding of what’s being built will lead to better estimates and a higher satisfaction rate from the stakeholders once the feature is complete.