Understanding data dashboards
data visualization

What is a Data Dashboard? Overview, Tips, & Examples

What is a dashboard?

A data dashboard is a live, self-service, exploratory tool that visualizes data to answer questions and help make decisions. Users of dashboards are interested in understanding the data presented in the dashboard. Because data can be complex, a key purpose of a dashboard is to simplify and guide the user to insights.

What is a dashboard in data analytics?

A dashboard is a key part of data analytics. Much of data engineering and data analytics has to do with moving data, cleaning it, shaping it, and combining it. But the tip of the iceberg is something that everyone can interact with – this is often the dashboard.

Dashboards visualize and simplify data, providing an intuitive exploratory interface.

Why do we need a dashboard to visualize data?

Data is complex, and the human brain is designed to understand visualizations and see patterns visually. Without dashboards or other ways to visualize data, people are left with spreadsheets and trying to navigate databases themselves. These approaches are much less effective, wasting so much time that people often do not even attempt them.

Data drives decisions, shaping the future of businesses every day. However, understanding this influx of information can be daunting, especially for senior business leaders who recognize the goldmine of data but grapple with unlocking its value. 

Advantages of data visualization

Benefits of Dashboard Visualization

User Friendly

Data analytics dashboards are powerful tools that are more than just dashboard software; they're the bridge that transforms raw data into actionable insights. Friendly to the user, reliable in delivery, and brimming with energy, dashboard solutions demystify data, making it both useful and valuable. 

Organize Raw Data

With a good data visualization dashboard, business leaders can see data in a new light, combining information from different sources to identify patterns and trends that were once hidden. It's not just about seeing numbers; it's about understanding what they mean in the broader business context.

Derive Business Insights

Whether you're a startup that's just secured funding or an established company seeking new growth avenues, dashboarding tools allow you to dive deep, exploring your data's true potential. The result? Unprecedented insights that drive passion for your brand and dedication to your customers.

Dashboard Software - The “How” to Data Visualization

What is dashboard software?

Dashboard software makes creating dashboards easier. These are often drag-and-drop tools that cover common visualizations. A user creating a dashboard can choose the metrics, filters, and groupings for the data, and then combine those with common visualizations like line charts or bar charts.

Is data dashboard software the best way to present information?

Sometimes, dashboards get a bad reputation. At its best, a dashboard is a live, self-service way for people to explore data, get insights, answer questions, and make decisions. The visual and exploratory nature of dashboards provides an experience unlike any other.

At their worst, dashboards are stale, confusing, bloated, and unused.

Like any technology, its application determines its usefulness and value.

Which software is best for a data dashboard?

Hundreds of dashboard software vendors exist. Some are very high quality, others have weaker offerings. None can be considered “best” because they all have different niches and strengths. 

The best way to evaluate is to gather as much information as you can, such as the Business Intelligence Product Evaluation Matrix by Datateer. Our matrix (a google sheet) takes all the most popular data analytic softwares and rates their features so that you can sort and see which tool may be most fitting for your business needs.

After, contact the vendors with products that best match your needs. Demos of these products immediately make clear where their strengths are.

How to make a great dashboard

3 Key Tips To Making A Great Dashboard 

1. Clean & Manage Data

The world of data can be a maze of numbers, charts, and graphs. But when harnessed right, these elements can come together to form a story, presenting fresh insights and overlooked opportunities. 

Most data isn’t static. Prior to being ingested by BI dashboards, automated data pipelines are created to clean data, check and remove duplicates, and monitor quality for freshness.

This allows senior leaders to tap into the real-time pulse of their operations, ensuring nothing slips through the cracks. The goal is to simplify the intricate, turning your data dashboard into a clear, dynamic, and valuable asset.

2. Pulling Data From Multiple Sources

Every data point captured is a reflection of a business moment. Over time, these moments paint a picture of growth, challenges, achievements, and aspirations. Yet, the true value of historical data isn't just in its existence, but in its accessibility and interpretation.

Creating a great dashboard involves not only curating this historical data but also integrating it from multiple sources. Your data analytics dashboard should serve as a nexus where past information converges, offering an expansive view of business trajectories. By pooling data from diverse sources, you provide a holistic dashboard data view that charts past performances and pinpoints future opportunities.

So, whether it's revisiting a successful marketing campaign from two years ago or comparing product performances across quarters, your dashboard platform should ensure you're equipped with all the knowledge needed to steer your business forward.

3. Shareability

In today's collaborative business environment, the ability to build and share insights is pivotal. But it's not just about having the data; it's about presenting it in a way that resonates, informs, and empowers. At Datateer, we understand the power of shared knowledge.

The first step is to build a data dashboards tailored to your specific business needs but don't stop there. Recognizing the need for collective decision-making, whichever dashboard solution you select should facilitate easy sharing of these dashboards. This ensures that whether it's a C-level executive or a director of operations, everyone is on the same page, leveraging the same insights.

Data dashboard examples

Examples of Different Types of Data Dashboards

Dashboards are most often created based around the user of the data or metric. For example, there’s rarely one “business” dashboard. But there could be an exclusive dashboard for executives who want to monitor data and metrics across every department. 

That executive dashboard is an amalgamation of the top KPIs for said departments. Each department may have its own dashboard for:

  • Sales
  • Customer service
  • Operations
  • Inventory
  • Marketing data
  • Etc

Conclusion

In today's fast-paced business landscape, the right dashboard tool can make all the difference. Tailored data assets, including dashboards, enable your team to tap into both current trends and historical insights, ensuring decisions are always informed. Every team member, from marketing to operations, can now harness data effectively, pinpointing opportunities and crafting strategies.

Datateer excels in creating and managing data dashboards tailored to each business's unique needs. Whether you're an ambitious startup or a seasoned business, our approach ensures that data is not just accumulated but is made actionable and shareable. With our expertise, everyone from C-level executives to operations directors can make informed decisions, grounded in clear insights. At Datateer, we don't merely offer dashboard software; we're experts in building and managing data assets. 

What Clients Say About Datateer Data Dashboard (Visualization) Services

"We created 100 different metrics very relevant to our customers. We have seen significant growth in our key accounts."

data analytics example

Paul Harty

Chief Strategy Officer @ Motion Recruitment

"Datateer understood our data and consolidated all that information in a way that dramatically improved the speed and quality of client conversations."

Devin Mulhern

Devin Mulhern

Managing Director @ Denver South Economic Development Partnership

Read More
spotlight on no-code, low code dashboards
Business Intelligence, Data Analytics, data visualization, Embedded Analytics

Spotlight on Low-Code and No-Code Dashboards and Datateer Partner Canvas

If you aren’t the most technologically savvy person, you may assume that the closest you’ll ever get to data analytics is the dashboards your analysts send you. Up until fairly recently, that was a correct assumption.

But with the advent of no-code and low-code analytics platforms, you can have a front-row seat in the creation of your analytics dashboards. More than a front-row seat, actually.

What Are No-Code vs. Low-Code BI Tools?

Low-code and no-code BI tools are similar in many ways but aren’t interchangeable. Instead, they each make building analytics platforms and dashboards an option for non-experts. Since you don’t need a data specialist or tech guru involved, these tools are a huge step in democratizing analytics.

No-code dashboards are precisely what they sound like; they don’t require any coding or SQL to set them up. Low-code dashboards, on the other hand, call for some minimal coding. Therefore, a low-code dashboard user needs at least a basic knowledge of coding language.

Both of these BI tools are part of rapid application development (RAD); they use automated code. Their drag-and-drop approach with pull-down menus means a nearly entirely visual interface for users. These formats are becoming popular in many organizations. More than 65% of app development will involve low- or no-code technology by 2024.

Yet, there are differences between these two approaches. Each has different users, different ways of being used, and different build speeds.

Pros of No-Code Dashboards

If you have zero programming experience but are interested in putting analytics dashboards together, you can put away your copy of Coding for Dummies. You won’t need it if you use a no-code dashboard.

Non-technical Maintenance

No-code dashboards are attractive for many reasons. The obvious draw for them is that you don’t need to rely on IT to set them up for you or adjust them every time something changes. Instead, almost anybody in the department can handle the dashboard maintenance. This way, more people can pitch in and be part of the process.

Easy to Learn

They require a minimal learning curve. There will be a bit of an onboarding process, but it’s simple and explanatory. Most people can just jump in and get started relatively quickly.

Fewer Mistakes

There is less room for error, too. Because the interface is all visual and you are only dragging and dropping items, these dashboards are pretty difficult to mess up. If only your kid’s birthday cake was as fool-proof!

Faster Time-to-Build

Don’t be surprised to discover that you are building your dashboards at a rapid rate. The no-code approach allows you to create your platforms up to ten times faster than the traditional, full-code way.

Cons of No-Code Dashboards

Sadly, just because you do away with coding, it doesn’t mean all your problems are solved. There are still a few shortcomings that you should be aware of if you go this route.

Limited Scalability

On the downside, no-code dashboards offer limited scalability. Their possibilities are somewhat limited; you have the options provided to you, and that’s it. There’s little room for customization.

Potential Security Risk

Another possible issue could be security. Without the guardrails of an IT expert in place, there may be a security risk in giving Bob from accounting the job of sharing potentially sensitive data.

Pros of Low-Code Dashboards

Low-code programming is pretty hot right now. And it’s easy to understand why. Low-code software is so popular that 77% of organizations currently employ it in some form, according to Mendix.

More Dashboards and Metric Customization

You get more dashboard and metric customization opportunities with low-code than with no-code dashboards. Your scalability is still limited compared to traditional methods, but you have more room to play with than you do with the very structured world of no-code dashboards.

Better Security

Low-code analytics platforms generally have better security. Because a person with at least slight technical knowledge is usually involved, they know which guardrails are best for protecting data.

Cons of Low-Code Dashboards

Of course, there are also drawbacks to low-code analytics. 

Less User-Friendly

As stated earlier, they may not be the most user-friendly option for completely non-technical users. A basic understanding of coding language is helpful. These systems aren’t designed for your average Joe but for developers. 

More Training

Low-code dashboards also require a bit more onboarding than no-code ones. Because they aren’t as straightforward, they do call for a little extra training before you can get running.

Slower Time-to-Build 

They are also slower to build than no-code platforms. There are more steps involved, so naturally, it’s bound to take a bit longer to go to market.

No-Code Dashboards from Canvas

Datateer is proud to partner with Canvas for their no-code dashboards. They offer pre-designed templates you can plug your metrics into, with the result that you get a better picture of your overall revenue data. 

Canvas knows the pain data analysts feel when trying to keep up with all the “small” changes needed to dashboards here and there; it all really adds up! So they took matters into their own hands; they said, “you do it, instead!” Canvas allows you to skip the coding class and connect to your data at high-speed.

Your metrics should be accessible to all, not guarded by the only people with tech knowledge. And it’s a trend that’s catching on. According to Gartner, there will be four times more untrained software developers by 2023 than professionals at large companies.

When Datateer partners with Canvas, you just pick your template and plug in your data. You can start making data-driven decisions from day one.

And Canvas takes security very seriously. Their servers are located only on US data centers which are SOC2 and ISO 27001 certified. So what’s that all mean? It means your information is secure.

Wrapping Up

The world of data analytics is coming closer and closer to your front door. It’s becoming practically essential practice for most companies to use their data to propel business forward. And the use of data visualization is a huge factor in helping companies understand their data. 

With technological advancements, more companies can have a more significant role in working directly with their data. The introduction of low- and no-code analytics platforms is an astonishing change, putting more data at more people’s fingertips. And when Datateer partners with Canvas, you get the best of all worlds.

Read More
A detailed look into the modern data stack
Business Intelligence, Data Analytics, data visualization

In-Depth Look Into The Modern Data Stack

A good analytics function in a business requires a combination of deep domain expertise, technical acumen, and data experience. Unfortunately, most of us lack at least one of those. 

This article is for leaders responsible for analytics in your organization, especially if you lack deep experience in technology or data. Inevitably you will have to make product and technology decisions. I want you to have a good mental model for modern analytics and a solid vocabulary to participate in conversations with vendors, consultants, data teams, and executives. 

The most prevalent and modern method of building an analytics platform nowadays is known as the “Modern Data Stack.” 

Data Analytics platform: a data solution that provides tools and technology from the beginning to the end of the life of the data. It includes data retrieval, storage, analytics, management, and visualization.

Modern Data Stack (MDS) Definition

The Modern Data Stack (MDS) refers to the cloud-based tools and software used to gather, organize, store, and transform data into workable business intelligence and dashboards.

At a high level, the Modern Data Stack should contain the following elements:

  • Best of breed (vs. a single product that does it all)
  • Scalable
  • Cloud-native
  • Centered on a data warehouse
  • Follows an ELT pattern

Cloud-native: a flexible way to store and structure data in a scalable way (allows for growth) and can be accessed quickly from anywhere. More than just an architecture for data, it’s an entire fully automated ecosystem for interacting with data.

Visualize the modern data stack

How to Visualize the Modern Data Stack

I like to visualize the Modern Data Stack as data moving in a left-to-right flow. The phrase “data pipeline” is often used, conjuring up images of oil pipelines and refineries. It’s a good analogy of the refinement process that data goes through as it progresses through these components. 

I’ll go into details on each of these components in a moment. For now, here is the overview:

Data Sources

On the far left are your data sources–anything that produces data that you need to analyze.

Data Lake

Next, we have the data lake, which is a landing zone for collecting raw data from the data sources. 

Data Warehouse

From the lake, data is transformed in the data warehouse.

Various processes (orchestration, observability, quality, support) plug into the warehouse in a general concept known collectively as DataOps (short for data operations).

Business Intelligence & Analytics

Once the data in the warehouse is transformed into an analytical data model, it is consumed by various audiences through data products appropriate for their needs. The most common data products companies start with are data dashboards or direct connections from spreadsheets. 

Components of the modern data stack

Deep-Dive Into The Components of the Modern Data Stack

Let’s go into more detail about these components. Something to keep in mind as you read through these–you don’t have to build these yourself. In fact, that is a poor practice with low ROI. For example, an intuitive approach is to build your own data connectors–”how hard could it be?” 

The best ROI is through combining specialized products and tools into a cohesive analytics platform. 

Data Sources

Data sources refer to anything that houses data you want to include in your analyses or metrics. They include things like:

  • SaaS applications like a CRM
  • ERP systems
  • In-house databases created by IT
  • Public data sources like census or weather data
  • Data brokers or providers who sell data
  • Spreadsheets that are manually maintained but critical to your operations

Connectors

Connectors are software tools that know how to do two things: extract data from sources and load data into destinations (typically the data lake or data warehouse).

This is a deceivingly complicated situation. First off, each connector is unique because each data source is unique. Imagine trying to create and maintain data connectors for all the thousands of SaaS products that are out there. 

Next, imagine trying to ensure you can support all the possible destinations.

It could be a nightmare. It is the single biggest challenge for vendors that specialize in data connectors. 

Data Lake

The data lake concept has been maligned and in some circles, fallen out of favor completely. But I don’t see it as optional. It serves three main purposes:

The Data Lake Acts As A Landing Zone

A data lake serves as a landing zone for collecting data from sources. Modern warehouses can perform this function–usually. 

If the connector you are using is robust at loading data directly into the warehouse, go for it! However, pushing data into a data lake is simpler, so fewer things can go wrong. The data lake also gives you a place to perform basic cleansing and preparation on the raw data.

Data Lake Gives Direct Access To Raw Data

More importantly but not as immediately obvious, the lake can be a place where data scientists can get direct access to raw data, where the original copy is guaranteed to be untouched and unmodified. 

Just having data from multiple sources in one place is a big benefit for data scientists or analysts who otherwise would have to do that raw collection themselves.

Separating The Lake From The Warehouse Reduces Vendor Lock-In

As you get into your analytics platform, you will quickly realize that the warehouse is the center of gravity and the biggest cost. The market will soon be dominated by a few large players who will continue to attempt to monetize you even more. Separating the lake functions from the warehouse will reduce vendor lock-in.

Data Warehouse

The data warehouse is the center of the universe in the Modern Data Stack. At its core, it is just a database. 

  • They support the SQL language, which is the ubiquitous way of querying data in databases. 
  • Tools that can connect to other databases usually can also connect to data warehouses.
  • Data is presented and interacted with in the well-established paradigms of tables, columns, and views.

An important distinction is that warehouses targeting the Modern Data Stack are cloud-native. They are designed for scale and priced on consumption. They can grow and grow to an unlimited scale (okay yes at some point there is a limit. But you aren’t Amazon or Walmart, right?). 

Data Warehouses Are The Biggest Cost In Your Modern Data Stack

I’ll bring up cost here because warehouses are almost always the main cost driver of your entire analytics platform. If you are familiar with SaaS products that typically have a monthly subscription, consumption pricing is next level. You are charged by the computing resources your operations consume, metered by the hour or sometimes down to the sub-second. 

The best analogy is your electricity bill. The more lights you turn on and devices you plug in, the more electricity you use, and the higher your bill will be. 

So although you are in control, there is no ceiling, and you are responsible for monitoring costs more closely than you may be accustomed to doing. 

Related Article: How Much Do Data Analytics Services Cost?

Orchestration

Orchestration refers to getting all these tools and products to play nicely together. The analogy is a literal music orchestra, with a conductor at the front keeping time and directing the flow.

Good tools and products exist for orchestration, but I have found this is an area where my teams spend significant engineering work.

Orchestration products provide the logic flow of what should happen and when, schedules or triggers to run programs that process data in the pipelines, and capture errors for triage and troubleshooting.

Data Products

Ultimately your audiences–whether they be employees, customers, or some other group–do not care about the technical underpinnings. They want access to the data. Data products are any asset you manage that allows your audiences to see and use the data. Typical examples include:

  • Dashboards
  • Visualizations
  • Reports
  • Notebooks
  • Data Apps
  • Datasets you expose directly to consumers (e.g. a view in a database could be a data product if you let people use it directly, but not all views in the database are data products)

As you go on your analytics journey, data products will proliferate. It’s natural. But only 20-50% of the data products you produce and manage will be valuable. That’s also just inevitable. Too many organizations treat them like permanent fixtures when they should be purging or redesigning.

It is also inevitable that in a living system, these data products must be maintained, or they will gradually decline in value. The thing to remember here is to treat them like assets and actively manage them.

Related Article: A Guide to Picking the Right Embedded Analytics Platform

Advanced Modern Data Stack Components

Any journey in analytics is a crawl-walk-run journey. Be wary of anyone who tries to sell you otherwise. Implementing the core components from above will get you huge value, even if some of the benefits of the advanced components are not in place. 

Artificial Intelligence or Machine Learning

Often abbreviated AI/ML, this simply means applying statistical methods and models to the data in your warehouse. For example, a popular application is applying linear algebra to historical data to predict future trends. 

Most leaders recognize the power that predictive analytics can have on their business. So, if there is a shortage in data talent in general, there is an extreme shortage in data scientists–people who have deep experience in a combination of your specific business domain, programming, working with data, and statistics. 

What is exciting is the number of vendors pursuing “low code” or “no code” solutions for AI/ML. While they will never surpass the capabilities of a true data scientist, they certainly deliver a lot of value for a much lower price tag. 

Governance

Governance answers the all-important question in business data, "Who can see what?" In other words:

  • what level of access does each user role have
  • what are you using for version control to keep track of modifications and access
  • what security certifications are required for your business data to keep your (and your clients') data safe

Data Observability and Data Quality

I lump these together because most products that focus here do a mix of both. Data quality is the goal, with observability being a mechanism to get there. 

Data quality is simply that the data can be trusted, is timely, is accurate, complete, etc. There are no industry standards for this. But it is a nice concept to use when audiences begin to mistrust data for a variety of reasons. Framing those conversations as data quality conversations can align everyone’s thinking.

Data observability is the concept of being able to see into the data pipelines and understand what is going on. This is especially helpful when there are errors or problems, but it is also useful for things like automatically detecting statistical differences in data. 

For example, if revenue spikes 100% overnight, that most likely is a data quality problem with a root cause that can be found through data observability tooling.

Embedded Analytics

One of the most valuable things a company can do with data is create insights for their customers as the audience. A popular method to do this is embedded analytics, which means creating a visualization or dashboard in a specialized tool, and then exposing that visualization or dashboard into a SaaS application so that end users can see it. 

This is not overly complicated but does require collaboration between product management, application engineering, and data teams.

Related Article: What is Embedded Analytics?

Reverse ETL or Operational Analytics

These are fancy-sounding terms that mean something quite simple. Once data is collected transformed, and made valuable in a warehouse, why not push it back out to the operational systems?

In this component, the data sources can become data destinations. Imagine gathering data about customers and orders from CRM, billing systems, customer support tickets, etc and creating a nice 360-degree picture of each customer. Wouldn’t it be nice to push that information back into the CRM, where the sales team lives and breathes? 

Streaming

Most analytics platforms at mid-sized companies are batch oriented. Daily data updates are the most prevalent pattern, but even updates every 10 minutes are considered batch updates–they are just smaller batches.

Batch: a system relied on for processing and analyzing vast quantities of data simultaneously. Applies to companies that hold their data in store for a period of time, as opposed to streaming data.

Streaming is a different animal and is required for true real-time or near-real-time operations. In general, the tooling for streaming is quite mature–but not for analytics platforms. Robust streaming tooling has been available to applications and operations like Internet of Things for a while and, in the coming years, analytics tooling will catch up.

Additional concepts

Additional Important MDS Concepts

Here are a few operational concepts and design patterns that you should be aware of even in an executive or completely non-technical role. You almost certainly will be asked to weigh in with your opinion to give guidance to the technical folks.

ELT vs. ETL

ETL stands for “extract, transform, load” and is a pattern that has been in place for decades. It is a data pipeline pattern that means extracting data from sources, transforming data, and loading data into a destination like a warehouse.

The problem with this pattern is it lumps the business logic of the data transformations together with the technical plumbing of moving data around. This may seem trivial, but it isn’t. 

By switching the order of operations–ELT or “extract, load, transform”--we can separate the plumbing of extract and load from the business logic of transform.

This leads to a much, much more flexible system and you should have a strong opinion here!

Combining Data Across Sources

If all you need to do is analyze data from a single data source, the Modern Data Stack is not the right solution for your business. 

The true power comes from bringing data together from multiple sources and doing analyses only made possible by being able to look across an entire business. 

I point this out because a common mistake is to expose raw data as a Data Product. If people get comfortable with the raw data, they will push back when you try to move them to use combined, transformed data. 

Worse, if your CRM tracks a real-world human by calling them a “contact,” but your billing system calls a human a “person,” then we have a terminology problem when it comes to analytics. Data efforts are already complicated enough, and terminology can be death by a thousand cuts.

Analytical Data Modeling

An analytical data model is an opinionated view of your business, made up of transformed data from multiple sources and designed for analysis at a large scale. I’ll break that down:

An opinionated view means you’ve reconciled terminology discrepancies and found (or forced) alignment on metrics and their definitions.

Data from multiple sources is not only brought together but converted into this opinionated view and combined. I.e. the “contact” and “person” records from above don’t live in different places in the warehouse but are merged together.

Designed for analysis means the structure of the tables and columns is designed to handle queries on large data sets, such as a metric that calculates average order size across your entire order history for the entire company.

Elevate your business intelligence team and increase profitability

Scaling

Scaling means growing or shrinking your system to support the workload at hand. Any cloud-based product vendor will tell you scaling is a solved problem. It isn’t, but the mechanics of scaling are way better than in the past.

If you are familiar with cloud platforms, you will be familiar with scaling. If not, here is a primer: before cloud, you had to run your systems on physical servers that you purchased. Buy a server that is too small, and your system (e.g., such as your website) might crash from too many users and too much load. So if you decide to buy a huge server guaranteed to support your users–you have overspent on capacity that you may never need. 

In the cloud, scaling is configurable. Sometimes you can manually decide how powerful to make a component; sometimes, it is automatic. In all cases, you pay for it, so tuning is required. 

You will be responsible for an analytics platform that is made up of many components, each with its own scaling approaches and limitations. Inevitably you will need an opinion on the tradeoffs between cost and high scale.

Emergent Systems and Iterative Implementation

Let me cut to the chase–you will not get all this right the first time. Historically, business intelligence or analytics efforts were huge projects that did not release anything out to the intended audiences for 12-18 months or longer.

We do not live in that world because the Modern Data Stack and its associated tooling, embrace fast change and iterative development. The result is business value earlier and ultimately more business value because the system is aligned with the needs and opportunities. 

However, the gravity of doing a big project is still very real. It manifests itself in many ways, such as phrases like “Go ahead and connect to data source X, we might need it” to “We can’t start building dashboards until the data model is done.”

Advice For Business Leaders Regarding the Modern Data Stack

Vet Each Modern Data Stack Tool Independently (Regardless of Partnerships)

If you are just starting to explore how to put together a good analytics platform for your business, it is extremely easy to get caught up in all of the products.

What tends to happen is as you start to select products or tools, those companies have a network of partners and will direct you to use their partner companies. It’s not necessarily a bad thing, but ask the question of why they partner with certain companies and why they passed on others. 

Things that are important to you will stick out, and it will help you build out a list of possible alternative products without getting sucked into the hype.

There Are Varying Levels Of Integration & Pluggability Between Vendors

Because the Modern Data Stack is a pseudo-standard in the industry, implementers and product vendors all understand this general blueprint. What they create generally fits with the standard blueprint, making things pluggable.

But because this is not a true industry standard, levels of integration and pluggability vary. Sometimes painfully so.

If you have been an owner or user of data warehouses in the past, be ready: operations are generally looser, the pace of change is (or can be) much faster, and containing cloud costs is super important.

When Hiring Talent Look For Someone With A Deep Understanding Of The Modern Data Stack

When hiring data people, one thing I look for is understanding and passion about the Modern Data Stack. I remember when the light bulb went off for me. It was the first time I saw software engineering best practices applied to data – a novel concept even today. It was eye-opening, and the possibilities were exhilarating. 

From there, it was a journey through dozens of Slack communities (each with thousands of members), reading opinion articles on how the Modern Data Stack should work, new conferences dedicated to pieces of the Modern Data Stack, and trying to keep up with the hundreds of companies receiving VC investment and promising to be the silver bullet to all the shortcomings of all data everywhere. 

Point being, there are a lot of newbies and a lot of excitement. But this is still a relatively new concept, so there is a lot of inexperience and mistakes. There is also a shortage of reliable talent with sufficient experience, and it’s getting worse. EY notes an increase of 50% in 2022 of executives focusing on data initiatives and increasing hiring.

Maybe the shortage will resolve itself as the tens of thousands of junior folks upskill and gain experience. But for the foreseeable future, demand is outpacing supply. 

Conclusion

Is the Modern Data Stack the only way to go? Certainly not, but it by far has the most traction compared to other approaches. The speed of development, shortened time to value, best-of-breed selection, flexibility, scalability, and usability of the Modern Data Stack results in a winning combination.

If you want a partner to help you navigate and manage your analytics journey, from crawl all the way to run, let’s talk about Datateer’s Managed Analytics Platform. Or if you want to learn more about this type of service, check out our Guide to Managed Analytics.

In future articles, I’ll discuss antipatterns to avoid, the Analytics Operating System to help you put the right processes on top of the Modern Data Stack architecture, and dig into benefits and drawbacks.

What Datateer Modern Data Stack Customers Say

"We created 100 different metrics very relevant to our customers. We have seen significant growth in our key accounts."

data analytics example

Paul Harty

Chief Strategy Officer @ Motion Recruitment

"Datateer understood our data and consolidated all that information in a way that dramatically improved the speed and quality of client conversations."

Devin Mulhern

Devin Mulhern

Managing Director @ Denver South Economic Development Partnership

Read More
Where Is Your Data
data strategy, data visualization

Data Security and Compliance in Cloud-Native Business Intelligence


This article is part of our series Selecting the Right Visualization Tool with Confidence. Other articles in the series include:


The risk of data breaches is huge, and is one of the main reasons companies are slow to adopt cloud computing. Facebook is in the news today for over half a billion profiles being leaked, including personal information! Google had to pay over $55,000,000 dollars in GDPR fines in 2020. 

If the big companies cannot get it right, why take the risk at all? Companies that do not become a data-driven business lose out to the ones that do. There is no sustainable alternative. Forrester research shows that organizations with data-driven insights are 140% more likely to create sustainable competitive advantage, and take tremendous market share from traditional organizations. 

In this article, my goal is to give you an overall understanding of what you should pay attention to and how to mitigate the risks. I am no lawyer, so you should make decisions based on advice from qualified legal, accounting, and security experts. 

Why use a cloud business intelligence tool?

This gets into a larger strategic question of digital transformation and whether to use cloud computing or infrastructure at all. In spite of all the benefits of using cloud infrastructure and SaaS tools, such as lower total cost of ownership (TCO), agility, and scalability, perception of higher security risk has been an impediment to cloud adoption. 

Ultimately, each organization has to make this decision on their own. Strategy in any risk-reward decision is greatly affected by how to mitigate or minimize the risks involved. Some organizations take a risk avoidance approach instead–which in my opinion outweighs the risk.

When it comes to using a cloud-based business intelligence tool, you will see much shorter onboarding time, and your maintenance costs are zero. So time to value is shorter, and if the tool’s pricing is in line with the market, TCO will be lower.

How is my data safe in an online system?

Data can be just as safe, if not safer, in an online tool than in a tool you manage internally. Seriously. In today’s information worker world, people connect into your network from home, coffee shops, mobile phones, etc. They are likely already connecting to your internal systems over a VPN connection, and to several SaaS products such as Salesforce or Jira. 

The top reason for data breaches is old, unpatched systems. When the security community identifies vulnerabilities in operating systems or network devices, they share these vulnerabilities in lists known as Common Vulnerabilities and Exposures (CVEs). This allows everyone to act quickly and with maximum information to resolve the vulnerabilities.

However, there is a catch. IT departments must actively manage servers, operating systems, and networks to apply these updates, so that they are no longer vulnerable. Although every IT department claims they are following best practices, the number of security breaches due to unpatched systems objectively states otherwise. Now, compare that to a product company, where every bit of their livelihood depends on keeping their SaaS tool patched and up to date–the incentives and human nature state are in their favor compared to your company’s internal systems

The second reason for data breaches is social engineering–tricking people into using weak passwords, sharing too much information, or providing an opening. Recent years have seen an 9-fold increase in these types of attacks, because of how easy and effective they are. These can be simple or complicated, but the methods attackers use often take bits of information from various sources to triangulate on a successful attack. Again, everyone assumes they would not be taken in, but the security research says otherwise. Using a cloud tool vs an on-premises tool does not impact this risk one way or the other.

You will likely run into more arguments against using an online business intelligence tool. With each, play devil’s advocate until you get past the platitudes and really understand how much of a risk each one might be.

How to evaluate a cloud BI tool’s security

Where is your data?

This is critical. How much of your data goes to the SaaS vendor’s servers? Two methods exist, with one carrying more risk than the other. In the first method, your raw data is brought into the vendors’ servers, where it is transformed or modeled into an analytical data model. In the second method, only aggregated data is brought into the vendors’ servers. The second is much lower risk, and most newer vendors take this approach.

This is a foundational architecture decision by the vendor, and not one they will be able to change for you. Often a vendor will tout this as an important feature–bring your data into our servers, and we can provide valuable data modeling to make your team more efficient (etc, etc). However, in the modern cloud data architecture, this is not a must-have feature. Tools like dbt are much more suited for this transformation, allowing your BI tool to focus on presenting the data, not transforming it.

A second question to ask about the location of your data is where the BI vendor’s servers are located. Most will be running on infrastructure provided by one of the big three cloud infrastructure providers (AWS, Azure, GCP), but not always. Each of these providers has regions globally. Depending on your industry, you may be required to guard against your data being “exported,” simply meaning that it cannot be transmitted or stored outside of your country. This leads to a line of questioning with the BI tool vendor about where their servers are located, and how they protect against data accidentally flowing through networks it should not.

Areas of exposure

When using a SaaS BI product, three technical vectors are the main consideration. When evaluating a tool, focusing on these three areas of security will be of most benefit: 

  1. The HTTP connection. This is the network opening that allows the user to connect from the browser. Do they use TLS/SSL for all connections? 
  2. The database connection. This is the network opening that allows the SaaS tool to connect to your data warehouse (which also could be on the cloud, or might be an on-premises database)
  3. Embedding. In an embedded visualization situation, this allows your application to embed dashboards from the vendor tool, and is important to review

Information security policies

You should ask to review these, and have a technology architect or security expert familiar with our business to call out any potential issues. Some of these will be technical in nature–encryption at rest, encryption in motion, etc. But the real focus of these is on people and policy. Things like password requirements; approvals; audits and reviews; and procedures and communications in case of breach. 

Mitigating and Minimizing Risk

Attestations and risk levels

Depending on your industry, you will have various regulations around data security. Some of the more widely recognized are HIPAA and GDPR. By using vendors who have participated in attestations or audits, you defer to experts and push much of the cost of risk mitigation onto the service provider. Here is how it works: consider a situation where a cloud vendor wants to process Personally Identifiable Information (PII) on behalf of its clients. Regulations state that these procedures must be audited. If each of the vendor’s clients must pass through an audit, that could mean hundreds of audits on the vendor, and each client must pay for their own audit. An attestation allows for the vendor to be audited a single time, and the auditor provides an attestation to each of the clients. This is efficient, cost effective, and is the standard in the audit world. 

SOC-2 is widely recognized as the standard for SaaS vendors. Developed by the AICPA, it is a robust framework that ensures a minimum level of compliance around data security controls. You should ask about this, as well as any industry-specific regulations and attestations that the vendor may have in place. Most of the companies we work with are not large enterprises, so working with one of the Big 4 accounting firms does not make sense for them. Linford & Co is the leading provider of attestations for SaaS companies and is a group I trust with these kinds of needs.

An up and coming risk mitigation strategy is to automatically monitor risk exposure of your vendors. The risk network curated by Cyber GRX. Not every vendor you are considering will be a part of this network. But if they are, using the information to reduce your third-party risk is an easy way to gain more comfort in using a particular vendor.

Alternatives to naive cloud architecture

In some cases, a hybrid approach between on-premises versus cloud is available. This is especially applicable in systems with a lot of moving components, such as a data platform. Datateer is designed for the security-conscious customer, with high levels of segregation to ensure there is no “cross-pollination” of data, and that data never leaves your control. This is not the mainstream approach, which is to have your data flow onto vendors’ servers. This makes things much easier for the vendor to process, but increases risk substantially.

One of the pillars in our stack is Prefect, which has pioneered this hybrid approach. This approach is more difficult for cloud-based BI tools to achieve. But as mentioned earlier in this article, if they have designed for it, they can prevent your raw data from flowing anywhere unnecessarily.

Part of this hybrid approach could mean hosting your own business intelligence tool on cloud infrastructure. This will guarantee that none of your data flows onto a cloud vendor’s systems, but it is quite a bit more maintenance. And it exposes you to the problem mentioned earlier of old, unpatched security vulnerabilities. Surprisingly, few options exist in this vein. Superset is a young but great option for internal analytics. And if you get going and realize managing your own solution is too much to take on, the project creators provide a commercially hosted option at Preset.

Insurance

Should be a no-brainer, but often this is overlooked. You and the vendor you choose should both have a Cyber Liability Policy including Data Breach Coverage. Regardless of whether you use a cloud vendor or on-premises solution, regardless of how good the information security policies and attestations make things seem, breaches are likely to occur. It is almost common knowledge that data breaches are a “not if, but when” situation. That doesn’t absolve all of us from our due diligence, but it certainly calls for protecting against the situation.

Summary

The benefits of using cloud-based business intelligence tools outweigh the risks. With a focus on mitigating and minimizing the risk, you can enjoy those benefits while protecting your business from the downsides.

In this article, we talked about key risks to be aware of and ways to evaluate BI tools in light of those risks. We also discussed ways to mitigate and minimize the risk of trusting a third party with your data.

Many vendors pay attention to all this and can help you understand the security posture of their products. You can also take advantage of Datateer’s free strategy sessions to talk through these risks and help make decisions.

Ultimately, the benefits will outweigh the risks for most, including you!

Read More
Visual Experience
data strategy, data visualization

Selecting the Right Visualization Tool with Confidence


This article is part of our series Selecting the Right Visualization Tool with Confidence. Other articles in the series include:


Evaluating business intelligence tools is exhausting.

First of all, there are a ton of them on the market. The high-profile acquisitions of Tableau, Looker, Chartio, and Qlik must be inspiring to entrepreneurs who want to have similar exits. The field is crowded. Even after years of discovering vendors and evaluating their products, we still discover new products regularly. 

Producing a basic tool–a UI on top of a visualization library–must be fairly easy, proven by how many are on the market right now. But building a solid product and company requires more than that. And so many nuances exist that can stop a report developer or designer in their tracks, or cause workarounds. How could you ever cover all those situations in your evaluation?

Every vendor’s message starts to sound the same, and is really some variation of how easy their tool will make the whole effort. Although 80% of the effort in data analytics efforts goes to data engineering and modeling, some BI tool vendors will tell you all of that can somehow magically go away.

From my experience, the BI tool is the tip of the spear or the top of the iceberg. It provides the visual culmination of all the work and analysis a company has put into their data platform–the presentation layer intended to produce something clean and useful. 

Really, more attention should be paid to getting the data, analysis, and metrics right. However, because the charts and visuals are what most users will interact with, the BI tool you choose is critical.

This article is the first in a series that will share our direct experience, the experience of our customers, and contributions from the community in the Chartio migration research project (here and here).

Below we lay out the overall approach we have developed to compare apples to apples. And in the coming weeks we will have articles diving even deeper into aspects of a BI tool evaluation we have found to be important:

  • Data Security & Compliance
  • Visualization Capabilities & Dashboarding
  • Self-Service Analysis and “Data Democratization”
  • The Support Experience
  • End User Experience (including Embedding)
  • Pricing and Budgeting
  • Performance
  • The Intangibles
  • Miscellaneous and Doing Too Much

A visual scoring system

In a crowded market you need some sort of way to compare things objectively. If you have ever had the experience of buying or renting a place to live, you have probably experienced house hunting fatigue. You look at so many homes, they all start to blur together. You can’t remember whether the kitchen you liked was a part of the first home you saw, or the second. Did you really like that 2nd house, or are you just tired when you see the 8th and want to make a decision?

That is the feeling we had when trying to make sure we were evaluating all the options. 

To combat this many people make a list of features or other aspects of their evaluation, and then create a scoring or rating system. We started that way, too. But numbers in a spreadsheet only go so far. 

Another thing we learned was this decision was not a one-time event. Each time we learned something new about any given tool, we found ourselves revisiting the spreadsheet to re-evaluate. We needed something that we could return to time and again, without wasting time rehashing things already discussed or re-orienting to a bunch of numbers.

We are evaluating a visual tool, so why not do something visual? By giving each feature two ratings–importance and score–we were able to create a simple visual experience. 

Here is a sample.

At a glance, we can:

  • see the overall value of one tool against a field of competitors. 
  • focus in on a single feature and see how each tool compares. 
  • immediately identify any missing critical features or holes in our analysis. 

Check out our Product Evaluation Matrix. Feel free to make a copy to get your own analysis started.

Not everything can be critical

Regardless of how you score or evaluate, each vendor is trying with all their might to be different from the others. So we end up with many features and variations of features, many of which are appealing.

But not everything can be of critical importance to your company. In a perfect world, you could enumerate the features you want, and someone would give you an order form and a price tag, and you are off to the races. But in an imperfect world like ours, you have to choose a tool that most closely matches your needs and desires.

It is the tendency of all us humans to overdo it and assume too many things are critical. At the extreme, if everything is equally weighted in your decision, you will end up with the most average product on the market, rather than the one that fits you best.

Recognize emotion and play the game

Buying decisions are emotional. Most of the decision is subconscious. According to the best research we have, emotion is what really drives the purchasing behaviors, and also, decision making in general. Experienced salespeople also know this, hence the saying, “sell the sizzle, not the steak.”

95% of thought, emotion, and learning occur in the unconscious mind–that is, without our awareness

Gerald Zaltman, How Customers Think

We use objectivity and logic to talk ourselves into things. Recognize this about yourself and the dynamics of the evaluation and decision process. 

Immediately after a good demo, you will have an affinity to the product that might stick. This is why every product company is willing to spend a lot of time demonstrating their product. Or, if the sales engineer had a bad day, it might cause you to see the product in less of a light.

To solve this problem:

  • Understand the sales process, and roll with it: first they qualify your company, budget, and timing; they push a demo; they provide statistics or other sales collateral based on your specific concerns; they give a free trial; they push for a decision; you negotiate terms and sign the 12 or 36 month contract. If you aren’t ready to move to the next stage, tell them. If they understand where they stand in your evaluation, often they will offer things to help you move forward, such as a longer trial period.
  • Eliminate early. If you limit your critical criteria and push those items in the early-stage calls, you can avoid spending time in product demos or trials for products that do not fit.
  • Pace yourself. Recognize that your deadline should drive the number of products that progress to your later stages. Give yourself time to evaluate, but set a deadline to force decision and action.
  • Use the evaluation spreadsheet. The visualization is intended to jar us visually–green good, red bad–and wake us up if we are being sucked in emotionally to a tool that objectively doesn’t hold up.
  • Review regularly. Have a regular review of your criteria and ratings with a decision committee. At least involve a trusted advisor or just a second set of eyes so that you can see things clearly and objectively. 

What happens if you make a bad decision?

I highlight this point specifically because we made the wrong choice prior to discovering Chartio, and had to live with the consequences. We also found that a completely exhaustive review of every single aspect of every single product that might just be the right one is totally impossible. Maybe you have that kind of time, and if so then use it. But if you are like us, you need to make a decision on somewhat incomplete information.

To mitigate the risk of a bad decision:

  • Extended trials. If you have the time to invest in deeper proof-of-concept exercises, many vendors will extend their trial period if they know they are in the running in a small field of competitors. Remember the sales stages and play the game.
  • Avoid doing too much in the tool. It is easy to forget that these feature-rich tools should be focused on presenting data. Doing too much data modeling or manipulation in the BI tool creates lock-in. Keeping these other responsibilities in your data warehouse prevents you from becoming overly dependent on the presentation tool. Focus on your critical criteria, and be selective about what makes something critical.
  • Remember the implementation cost. Costs of standing up a new BI tool are usually about the same as the cost of the tool itself. So plan for 2x whatever the price tag before you will see any return on investment from the tool.
  • Have backup plans. If the tool does not deliver as advertised, what then? Often in business there are less optimal ways to accomplish something that you can use as a plan B. For example, we often embed custom visualizations. Our plan B was modifying our custom visualization server to overcome some limitations of a product we chose. Our costs increased, but it wasn’t the end of the world.
Read More
HubSpot Tracking Leads
data visualization

Do Something More Interesting with Your Data than Internal Reports

We have arrived at a time when data is finally treated not as a technical thing but as a business asset with inherent value. Business leaders who are not tech professionals now understand how powerful data can be when harnessed and applied. Technologies for processing, analyzing, and delivering insights and analytics have progressed in leaps and bounds.

So why are most companies still stuck in a data landfill? They have plenty of data, but it analytics are disorganized and most data just sits there, unused. Forrester says up to 73% of data is never even touched for analytics. While the limits of what is possible expand rapidly, execution is not keeping pace. Most business leaders I know have ambitions, a vision, and plenty of data at their disposal — but get stuck trying to bridge the gap between current state and their goals. Plus, data creation is increasing exponentially, making the situation worse.

Companies are stuck in this state because traditionally data efforts are aimed at improving operations — looking internally. This kind of effort is typically characterized by connecting all your operational systems, combining data into a big data warehouse, and putting a BI tool on top to produce reports. Management uses these to analyze operational performance and seeks to incrementally improve performance. These are usually large, expensive, risky projects that take a long time to realize any return on the investment.

Contrast that with customer-facing analytics — efforts targeted at your customers or users of your service. These have a broad audience, can start small, and differentiate your product from the competition. By differentiating an offering which involves analytics your customers care about there are immediate effects to top line revenue. I’ve observed these companies increasing win rate, growing key accounts, and improving customer retention.

What does a customer-facing analytic look like? Often it can be simple visualizations that answer questions your customers care about. Are they using your service efficiently? Are you delivering on your SLAs or promises? Can you provide some strategic insight?

Here is an example from HubSpot, which allows their users to go beyond tracking individual leads into understanding their sales funnels as a whole and spot trends:

LinkedIn proves that dashboards aren’t always necessary, with a single chart to help understand traffic to a company’s LinkedIn page:

My favorites are subtly embedded analytics or visualizations, tucked into the application itself rather than as a stand-alone dashboard. Here is another example from LinkedIn, showing immediate feedback on posts and other actions a user takes:

Of course, customer-facing analytics go well beyond visualizations. Ultimately, if you can answer important questions for your customers using visuals, predictive algorithms, in-person discussions, or plain old excel exports, you are going to improve your relationship with them and their dependency on your product or service.

Read More