Chatting with Data: Interrogative UIs for Exploratory Data Analytics

So what’s an interrogative UI?

in·ter·rog·a·tive

/ˌin(t)əˈräɡədiv/

adjective

1. having or conveying the force of a question.
   

Interrogative UIs are essentially chatbots, voice command interfaces or any other user interfaces that are designed specifically to field questions expressed in natural language. Conversational user interfaces that can process interrogative input (aka questions) have long since been ubiquitous in mainstream, consumer products (Siri, Alexa, Google, customer services chatbots…)  but have been slower to be adopted in enterprise applications. So it is reasonable to wonder why these methods are not employed more in a business setting, where so many workflows rely on querying data driven systems.

So what’s data exploration?

Data exploration is an approach similar to initial data analysis, whereby a data analyst uses visual exploration to understand what is in a dataset and the characteristics of the data, rather than through traditional data management systems - Wikipedia

One of the most valuable outcomes of exploratory data analysis is that it reveals trends and relationships in data that are not readily apparent. This neatly falls into use cases committed to discovering unknown unknowns but can be incredibly hard to perform on large scale, highly connected data. The amount of drill-down, filtering and pivoting needed often presents challenges that make this type of analysis difficult through conventional means of exploration. Data exploration at scale usually means synthesizing data into complicated visualizations so trends, correlations and hidden relationships can be revealed. Sometimes visibility across vast datasets can, often, only effectively be achieved through visualization. Human beings tend to lack the cognitive capacity to explore complex, highly connected data through a series of spreadsheets. And once any interesting information is surfaced, sharing it in a readily understandable and impactful way, quite often, includes visualization.

Conversations with data

Questions instead of queries
Data exploration requires trial and error, hypothesis testing, comparison and other circuitous paths through data to find meaningful information. Long strings of nested logic and complex joins make writing queries a poor proxy to represent the thought process that one goes through to derive meaning through data exploration. Dashboards with complex filtering and deep drill-downs also fall short when complex data exploration is considered. Technologies such as Python notebooks fare slightly better, but this takes a particular set of advanced skills and time to unlock their power. So, what if we could forgo the overhead of all the imperfect proxies that stand between our questions and the answers we are looking for?

The natural language of investigation is interrogation - asking a question, getting an answer, and asking a followup question we form based on the outcome of that answer. The natural language of interrogation is, well, natural language. Deductive and inductive reasoning is the cornerstone of how we organize our brains to perform exploration, and we think through this, chiefly, in natural language. Why force someone to then map this logic (formed in natural language) to a set of filters or a long, complicated nested query?

Data visualizations as answers

Verbal feedback is usually the expected outcome of a verbal question. For data exploration, however, this is usually not enough.  As stated above, data visualization is sometimes the only way to gain visibility and insights when working with large, complex datasets. Coupling natural language answers with visualizations alleviates issues traditional data analysis tools struggle with. Analytics software often presents data in a series of charts and graphs in some kind of tiled dashboard format. But important information and correlations between visualizations are often a result of a succession of complex filter combinations, pivots and drill-downs.  The user has no logical roadmap that plots out how s/he arrived at a given outcome. Showing visualizations in the context of a conversation, allows the user to relate visualizations back to a logical progression.  Steps and be reviewed and alternate paths can be explored without losing prior paths of exploration. Results can easily be reproduced and thought processes can be shared through collaboration.  In these cases, context is everything and  journey to insights can be as valuable as the insights themselves.  

Recommendations and guidance

Benefits of gaining information through a healthy reciprocal conversation, is the ability to clarify and proactively suggest alternative or additional information. This happens naturally and almost subconsciously when two humans converse. Conversational UI design has embraced these language patterns and has incorporated them into user experience strategies to mitigate workflow dead ends such as no search results or query syntax errors. This linguistic tool is especially powerful guiding users to discover the “unknown unknowns” when performing data exploration. 

Conclusion

‍
Interrogative patterns and conversational UIs are an established way for people to interact and query data. This is mainly utilized in the consumer application space but can easily be extended to data analytics and data exploration use cases in enterprise domains. Interrogative UI, coupled with data visualization, enhances the ability to contextualize data exploration in a way that more conventional data analytics methods cannot. These are all techniques employed by our multidisciplinary data analytics practice at Expero. The key to our success with such endeavors is an expert understanding of both user experience and the technical underpinnings of innovative user interfaces in data analytics.