This article is part of our series Selecting the Right Visualization Tool with Confidence. Other articles in the series include:
- Selecting the Right Visualization Tool with Confidence
- Data Security and Compliance in Cloud-Native Business Intelligence
- Discovering the Analyst Experience and Impact on Data Democratization
Data democratization is definitely a buzzword. Like digital transformation and moving to the cloud, it is ambiguous and can mean different things to different companies. But like many buzzwords, it contains a nugget of truth. Data democratization is truly a shift. When choosing a cloud business intelligence product, it is important that the vendor’s perspective aligns with your own. Otherwise you will end up with that familiar situation that the product does not quite “fit” with your organization.
What is data democratization?
Some years ago, I was part of a successful data warehouse initiative at a large financial services institution. Overall the project had gone very smoothly, and we were able to produce analytics that did a good job answering questions for the business. However, I became aware of an interesting pattern. We began to receive more and more requests to produce answers to questions, rather than the business users answering their own questions. Folks were willing to wait in the queue for days rather than attempt to answer their own questions with the tooling provided. It was just easier for them to ask the data team.
This experience illustrates the essence of data democratization. Although we had produced a good data warehouse, we had failed to create tooling that enabled everyone to participate in answering their own questions. Here is a deeper explanation from Forbes.
The rise of the Data Analyst
No matter how many slick marketing campaigns tell you their product makes data easy, working with data is hard. Quanthub describes the skyrocketing demand for data analysts, happening because they are the bridge between data and knowledge.
(P.S. The term “data science” is all the rage, and you may be tempted to go hire a data scientist. But data analysis is where most answers come from and is much more generally applicable).
The ideal state is that anyone in your business can answer their own questions. However, the current reality is that many users are not comfortable working with data and operate more like consumers of reports and analyses. But there is a major shift happening right now: more people can function as analysts, one reason being that new tools are providing new capabilities to support this.
Here are fundamental roles on a modern data team, to highlight the change in the data analyst role:
- Data Architect. Defines the overall system and how all the pieces fit together. Defines the data model in the warehouse
- Data Engineer. Writes code to move data from sources to warehouse, combine data from multiple sources, transform data so it fits into the warehouse model.
- Data Operations. Monitors and maintains the live system
- Data Analyst. The connection between the business and the data. Analyzes the data to produce answers to business questions
- Data Scientist. Applies statistical analysis on data to test hypotheses and create predictive models.
The major shift in the data analyst role is they are no longer part of the data team. And this has nothing to do with a title or formal responsibility. More often, they are part of business departments, and they happen to have an interest and ability to understand the data. You may have encountered the person on the marketing team, for example, that is a spreadsheet maven and always seems to have a spreadsheet available. Or the person on the finance team who produces charts and graphs in presentations that show just how the company is doing.
Many companies are embracing this analyst-first reality and looking for tools that can support this mode of operating.
Buzzwords versus reality
The buzzword folks promote a vision where everyone is working with data. The reality is that the analysts have become the true workhorses, and are the bridge to everyone in your organization successfully using data.
Adopting tools and processes that fit this reality will be of most benefit to your company. From our research, many vendors do recognize this reality and take varying approaches to supporting the data analyst.
Various approaches to the analyst experience
One benefit to a crowded BI market is that it forces innovation. Below are some various approaches to the analyst experience.
Separate the query from analysis
A downfall of earlier tools (and I am only talking a few years ago, not to mention decades ago) is they assumed all users knew SQL. Most analysts come from a background using spreadsheets to analyze data. Thus, converting the data warehouse tables into datasets that feel more like spreadsheets is going to enable more people to use the data.
Sigma Computing is a company that has embraced this concept. Not only do they separate the data set generation from the analysis, they are all in on spreadsheet analysis. In evaluating their product, I found this intuitive and right in line with an analyst-first approach.
Holistics also pushes the approach of providing modeling separate from analysis. This allows the more technical data team to define datasets, opening up the analysis to a broader group of people.
Panintelligence is an embed-first tool that follows a similar approach of defining a model that it then uses to drive GUI-based creation of visualizations.
Each of these uses the information in the model to generate queries that leverage the data warehouse infrastructure. Some tools actually process the queries on the BI tool’s infrastructure, which can be a data compliance risk. Be sure to ask.
Allowing people who do not know SQL an ability to generate queries (rather than write them directly) has been a long time coming. The idea is not new, but only in the last few years have vendors been able to do it well. Like bumpers in bowling, this approach allows more people to use the database without ending up in the gutter.
Trevor.io leads with their query builder functionality, and they say this is where most of their users spend their time.
Chartio had really begun pushing this with their Visual SQL feature, before their acquisition.
Although this is a great approach and an improvement over requiring all users to know SQL, there is often a tradeoff. The easier a tool makes this for the user, the less flexible it becomes in the types of queries and analyses that are available. When evaluating tools that take this approach, be sure to understand how well they support direct SQL access, and what the tipping point is where getting into SQL is necessary.
This is an earlier approach that requires analysts to know SQL to make queries to the database. This is still powerful, especially if the analysts in your organization do know SQL well. This approach allows for a lighter-weight tool that can be quicker to deploy and use. It does, however, come with another tradeoff in that metrics and analyses will become inconsistent over time.
Metabase is a tool we have enjoyed using. They do have some query builder type of features, but we found that we quickly jumped into SQL in most cases.
Redash is very lightweight and essentially just a visualization tool on top of a SQL database. For someone who knows SQL, Redash is easy to get into and start using.
We were a customer of Mode before moving to Chartio. Because Mode takes a direct SQL approach, it is well suited for larger data teams that need good collaboration among each other. But it is intimidating for analysts with a less technical background.
Preset actually takes the direct SQL to its logical conclusion, with SQL in everything–queries, metrics, filters, formatters, etc. It is very powerful, but obviously requires some SQL expertise.
When evaluating business intelligence tools, recognizing the approach they take to the analyst experience is crucial. No matter how good your data pipelines, data warehouse, and other pieces of your platform are, the analyst is going to make your data initiative successful. This article discussed how to understand the data analyst in the overall process, as well as various approaches vendors take to providing a good analyst experience.