What is the Modern Data Stack? A Quick Guide for Non-Technical Professionals

Let’s break down the modern data stack and what it means for normal business people (i.e., not data engineers).

Defining the “Stack”

What is a “stack” anyway? Simply put, it’s a collection of software working together. Data flows through these systems, hence the term “stack.” Data analytics typically involves 3-5 or more pieces of software working in harmony.

Understanding the Old Data Stack

The old data stack refers to the “mainframe” or “on premises (on prem)” server that physically sat in a company’s office building. It required extensive configuration of software on local servers, computers, and networks.

What is the Modern Data Stack?

The modern data stack (MDS) refers to the group of 3-5 cloud-based hosting and software tools needed to gather, organize, store, and transform raw data into usable business intelligence for SMB and enterprise businesses. 

Like much of the software you use today, it has moved to “the cloud.”  This means all your raw data, the data analytics, and the software needed to merge, analyze, visualize, and interpret it resides on someone else’s servers, rented for a monthly fee, of course.

Four Key Components Of The Modern Data Stack:

1. Data Sources: Gathering the Raw Material

Data sources are where your data originates for instance your website, app, or social media platforms. This includes user data from your company’s software, sales data from Salesforce, marketing data from Hubspot, and finance data from Netsuite, combined with individual Excel or Google Spreadsheets used in each department of your business.

2. Data Warehouses: Organizing and Storing Data

Data warehouses are the repositories where your cleaned and structured data resides. Major players include Snowflake, Amazon Redshift, Google BigQuery, Microsoft Azure, and IBM Db2. Data engineers use various software for ETL (Extract, Transform, Load) to move data between sources and warehouses, create automated pipelines to ensure smooth data flows, and monitor those pipelines for issues.

Related Article: Data Lake vs Data Warehouse vs Data Mart

3. Metrics: KPIs that Matter

Metrics help you understand performance, track trends, and make informed decisions. Tools allow you to visualize data, create data dashboards, and explore insights. For example, customer churn measured month over month may combine raw data from your software platform, finance data, and customer success data. Metrics can even be non-numerical, abstracting raw data into key indicators like high or low.

4. BI Tools: Analyzing and Visualizing Data

BI (Business Intelligence) tools, including data analytics platforms, help analyze and visualize data to extract insights. Tools like Tableau, Sigma, Microsoft Power BI, or Looker create dashboards with charts and graphs for meetings, sales presentations, and strategy sessions. Some excel at handling large datasets, others are user-friendly for non-technical users, and some are designed for embedding analytics into websites, ERPs, or mobile apps.

Related Article: A Guide to Picking the Right Embedded Analytics Platform

Modern Data Stack Diagram

In the diagram below you can see how we, at Datateer, use the modern data stack to transform our client’s raw data into usable reports and dashboards.

Conclusion

In summary, the abundance of data analytics vendors stems from diverse organizational needs, technological advancements, market fragmentation, and evolving business models. While this offers ample choice, it can be challenging to navigate.

Systems integrators like Datateer exist to simplify the process for normal business people. Just provide your sources and KPIs, and we’ll handle the rest. After all, you’re the business expert; we’re the data engineers who find this stuff cool.