Everything Wrong With Analytics Dashboards (And Our Plan to Fix It)

I was in charge of analytics and reporting when I joined LANDR Audio, where Ludo and I used to work.

For the longest time, we didn’t have any dashboards (startup!). We had TVs, but there was nothing on display.

This was until Ludo got fed up one evening, and decided to connect Mixpanel data to Dashing.io to create a dashboard of our core business metrics .

After setting up a few Raspberry Pi-s in the office, suddenly there was light ?:

The team was pretty impressed:

Life with an Analytics Dashboard

For some in the 50+ employee company, it was their first time seeing the data.

For others, the data was just a subset of what they were used to look at.

And for most, it was not exactly the data points they were hoping to see…

Although the initial reaction was really positive, it took less than a week for the change requests to roll in:

• Could we see this as a ratio instead?
• Can we calculate this number differently?
• Can we add a sub-view for that chart?

What began as a simple burst of enthusiasm soon grew in scope and complexity; it was hard to reconcile the different needs, perspectives and worldviews.

From what we saw in dozens of businesses, this was not an isolated incident.

The Problem With Analytics Dashboards

Dashboards are subjective in nature.

They’re a person / team’s opinion of what matters to the business.. and what matters always depends on the goals.

When people are saying: “I’d like to see this or that number instead”, they’re often challenging the dashboard’s purpose.

Is the goal of your dashboard to…

• …drive action?
• …track performance against the business objectives?
• …show progress to investors?
• …monitor KPI health and performance?
• …make people feel good about how the business is progressing?

These goals all lead to the creation of different dashboards. Those dashboards won’t be wrong, they’ll just tell different stories.

By default, analytics dashboards say nothing about their goal or purpose; they’re blank canvases populated by logical data points (usually ?).

Dashboards rarely sustain engagement in businesses because everyone has a unique point of view. They tend to fail because the business does not acknowledge their subjectivity and purpose.

The Analytics Dashboard Landscape

Businesses have access to an ever-increasing amount of data — there’s data in the database, the CRM, the marketing automation platform, Google Analytics, the financial dashboards etc.

The challenge however is to make sure that decisions get made with the best available information and data anywhere in the organization.

There’s a lot of great dashboarding products out there (Geckoboard, Klipfolio, Databox, Google Studio, etc). These tools are flexible, powerful and have tons of integrations and connectors; they make it easy to create custom dashboards per teams, products, business units, etc.

What businesses gain in flexibility with these tools, they lose in clarity. A general purpose dashboard fails to communicate the context behind the data points.

Key Gaps in Analytics Dashboards

More so, analytics dashboards:

• Assume knowledge: Any analytics, sales or SEO dashboard implementation presumes that the people that created the implementation understand what makes good metrics, what they’re putting on display, and how they can avoid gaps that stem from the data’s subjectivity. Sadly, that’s not always the case. Selecting metrics to prop up often means creating blindsides for your business.
• Don’t understand context: The same numbers will be interpretated differently depending on the role, responsibilities or business unit of the person looking at it. A monthly churn rate of 5% won’t mean the same thing to a product manager as it will to a support specialist. That’s just normal. With many data platforms connected to a dashboard, it’s easy to lose the context.
• Provide incomplete information: No matter how well designed a dashboard was, the data still needs to be interpreted in order for people to get insights. A dashboard shows charts and numbers, it does not turn these numbers into knowledge or explanations (Highlights does ?).
• Require monitoring: Now, what happens if a key stat starts dropping? Some dashboarding platforms will allow you to create alerts for different thresholds, but the creation of those thresholds already requires determining what’s good and what’s bad. Most alert systems tend trigger too many alerts and false positives. This creates de-sensitization to alert messages (see: Boy Cried the Wolf).
• May promote vanity metrics: Vanity metrics are metrics that make you feel good without telling you anything about your business. Analytics dashboards won’t tell you whether the metrics you’re betting your business on are vanity metrics or not; it just won’t know.
• Don’t give good insights cross-tools: Analytics dashboards are designed to connect to tools one after the other. They’re good at displaying specific data points, but not at combining data sources to create new (calculated) metrics or insights (e.g. Cost of Acquisition Payback).

Although dashboarding tools can do wonders when businesses have sufficient data maturity, for most organizations it won’t drive the right behaviors.

To that end, specialized dashboards often beat general purpose dashboards.

Moving Analytics Dashboards Forward

Information – so abundant it's worth practically nothing.

Knowledge – scarce enough to be worth $, price determined by supply & demand, distribution follows a bell curve.

Insight – So rare it could be worth a fortune, distribution is fat-tailed and the Black Swan rules.

— Johnny (@Johnny_Uzan) August 21, 2018

According to ex-US Secretary of Defense Donald Rumsfeld, knowledge has 4 dimensions:

Analytics dashboards should work with all of the 4 dimensions. Instead of reporting numbers, they should communicate insights into:

• Known Knowns: The known knowns are your business drivers. The KPIs and metrics your business needs to understand whether or not it’s operating properly. These metrics may include your North Star metric and most of your core KPIs.
• Known Unknowns: The known unknowns are key tests and experiments. They’re gaps you or your team have identified and wish to make progress on. You want to keep a close watch on these experiments before you integrate the findings into your model (known knowns).
• Unknown Unknowns: Unknown unknowns are the things you don’t know you don’t know. They’re the silent killers. ? Although it’s probably impossible to de-risk all the unknown unknowns of your business, sometimes you only need a different set of expertise to uncover the gaps in your organizational knowledge.
• Unknown Knowns: Unknown knowns are the things you once knew, but may have forgot. They operate like unknown unknowns, but lead to different action items. Once you’ve recognized a familiar situation, you can add them to your known knowns. Examples from this category are seasonality or other patterns you’ve already been made aware of in the past.

Our initial vision for Highlights was to augment existing analytics data with growth and internationalization insights (see: Two Pivots And Three Countries Later… We Have an App). We wanted to help businesses avoid blindspots like:

• Payment issues;
• Slow decreases in traffic or landing page conversion;
• New consumption patterns;
• Site performance (page load time, user experience, mobile, etc);
• Etc.

What we quickly realized though is that dashboards are only a delivery platform.

Complete analytics dashboards should operate along Rumsfeld’s 4 dimensions of knowledge. This realization led us to turn Highlights into a status system for inbound marketing programs.

Because Highlights is specialized (it constantly analyzes digital marketing data), it has a easier time identifying gaps, leveraging knowledge from hundreds of similar situations and growing performance across our user base.

Analytics dashboards need to evolve from passive to proactive platforms and monitoring systems. We’re already making headways with Highlights’ smart executive dashboard, and we expect dashboarding platforms to follow in our footsteps.

It’s 2019… Let’s make analytics dashboards great for good! ?