BI’s third form

Help us with the hard thing.

We can read the signs, but we don’t know what they mean.

Ten years ago, if you were a jaded forty-something-year-old tech executive, you bought a Porsche and a house in Jackson. You opened a bar in Oakland, or a coffee shop in Mill Valley. You learned a lot about woodworking, Patek Philippe, and divorces. You Benjamin Buttoned yourself—got hair plugs, dressed down by two generations, and met your new 28-year-old fiancée in the middle.¹ 

Today, there are new fads. Today, every jaded forty-something-year-old tech executive has one of three ambitions:

1. Buy some HVAC businesses. Or laundromats. Or family dental offices. “Tech startups are all childish hype,” you say. “They’re kids playing company. Did you know that car washes have a 12 percent EBITDA margin, but are valued at only 3x revenue? I’m raising a $25 million SPV to roll up all the regional car washes in north Florida,² streamline back office redundancies, optimize their digital marketing strategies, introduce a new recurring subscription service, and leverage AI.” You tell your investors that you have a network of experienced industry veterans as advisors, but you know that you won’t need them. You were the CRO of an enterprise infrastructure security company that sold to half of the Fortune 500; how hard can car washes be?³

2. Start a venture studio. “I’ve got a lot of great ideas, but my time is too valuable to gamble it all on just one company. Instead, I’ll incubate a bunch of startups in a venture studio, figure out the best one, and then hire a young, ambitious CEO to take it over. I’m more of a zero-to-one founder; my superpower is strategy and product vision. It’s not that I don’t want to do the work—Jensen Huang and I have both been saying this for years—it’s just that it would be inefficient.”⁴

3. Bootstrap an AI startup. “Did you see Sam Altman say that somebody will soon build a one-person billion-dollar company? I wasn’t going to do another startup, but AI changes everything. In my twenties, I cared too much about fundraising announcements and which VC invited me to which parties. That stuff is all a distraction. This time, I want to keep the company lean, focus on profitability from the beginning, and own as much of it as I can. I don’t know what I’m going to build yet, but ChatGPT and I are talking through it.”⁵

For most people, these are the inevitable attractions. However, if you’re a jaded forty-something-year-old tech executive from a data company, you might be tempted by a fourth option: Build a data product, for a specific vertical. 

For you, after having spent years making operational middleware for data teams, the grass (and money) in other departments is starting to look greener. “Analytics teams are cost centers,” you might say, “Their budgets get cut too quickly. We have to sell customers on our product and on the value of their data teams. Rather than making yet another BI tool, wouldn’t it be easier to create an analytics application for ecommerce marketing departments? Or a sales forecasting app? I want to get back to the basics—helping the actual business people who provide actual business value.” 

In the analytics market, this our version of the bundling and unbundling pendulum. We make generic BI tools; we make department-specific applications. When we get tired of making “custom tools [that] help others look at very narrow, specific datasets,” we make more generic BI tools; then we make department-specific applications again. 

Over most of the last decade, general-purpose tools were the cool thing to build. Rather than tracking, storing, and reporting on product usage in Mixpanel and tracking, storing, and reporting on sales metrics in Salesforce, we tried to store all our data in one centralized warehouse, and report on all of our data using a universal analytics layer. This was one of the motivating beliefs behind the modern data stack: Pivot the vertically-integrated tools sideways, and replace them with horizontal platforms.

Ah, whoops. So now we’re swinging back the other way, and business apps are trendy again. Between 2023 and 2024, Matt Turck’s data landscape grew by 42 percent, from 1,416 companies to 2,011 companies. The general-purpose analytics categories—BI, data visualization, and data analyst platforms—grew by only 26 percent, but domain-specific categories—product, sales, marketing, finance, HR, legal, and customer experience—grew by 56 percent.⁶ 

It’s the circle of life in Silicon Valley: Bundle, unbundle; centralize, decentralize; pivot horizontally, pivot vertically. Build multipurpose analytics and BI platforms that are sold to data teams; build domain-specific tools that prestructure common departmental data sources and predefine popular business metrics. Do your analysis freehand; trace a ready-made template. Alternate between the two forms of BI. Though each cycle works a little better than the last, nothing fundamentally changes. 

Generally accepted analytics principles

There’s another way to describe this oscillation. A friend of mine⁷ has a theory that all enterprise software should do one of two things: 

1. Give people tools to creatively express their ideas, and enable them to be experts in their craft. 

2. Keep them from screwing up.

Final Cut Pro and Photoshop are the first type of tool. They make it (relatively) easy for filmmakers and artists to turn the ideas in their head into images on a screen. But Final Cut doesn’t tell editors how to make those images, and Photoshop doesn’t box in designers around some Adobe-imposed best practices. These tools, and tons more like them, are microphones for their users’ skills—they both amplify greatness, and are unforgiving of mistakes.

We use the second type of software for protection. People need to pay our taxes; TurboTax fills out their forms and makes sure they’re right (powered by TurboTax® CompleteCheck™!). Managers need to keep their teams aligned around a set of goals; Lattice not only helps them track those goals, but encourages that they do it in a very particular way. If you try to use Lattice to manage people using some deranged goal-setting framework instead of one they sanctioned, the tool will get in your way. And that’s the point—to tell us what to do, and protect us from our creative flights of fancy. 

In the data market, open-ended products like Mode, Hex, and even Excel fit into the first category. These tools give analysts code editors and spreadsheets, and let people do whatever they want with them—model quarterly discounted future cash flows, paint pictures, nuke the global economy, whatever. They’re products for quantitative creativity. 

BI, however, is more complicated. Most BI tools have two major components: first, a data model in which people define metrics, and how they should be calculated; and second, an interface where people choose which metrics they want to see and how they want to visualize them. The former thing defines the rules that govern the latter thing. 

This creates two different experiences, depending on who you are:

1. If you’re the person who’s trying to answer questions—like an executive who wants to know how much money you made last quarter, or a marketer who wants to see how an ad campaign is performing—BI tools are designed to keep you from messing up. You can only look at a list of predefined metrics, and you can only explore those metrics in pre-approved ways. Just as Lattice is meant to keep managers from improvising how they set goals, BI tools are designed to keep people from creatively expressing what revenue means to them. There are lines, and you have to color in them. 

2. If you’re the person defining the metrics, however, you can do whatever you want. If you want to define your financial metrics in an insane way, your BI tool probably won’t nudge you toward a more generally accepted definition.⁸ Nobody is watching the watchmen. If an analyst wants to creatively express what revenue means to them, they can.

Most domain-specific templates and data tools—the stuff we’re flirting with again, now—transform the second “creative” experience into a more governed, “protected” experience. For example, Google Analytics defines sessions automatically, according to some unchangeable internal logic. PostHog, a product analytics tool, calculates retention cohorts based on some methodology of their choosing. SaaSGrid has “all the pre-built transformations you need: Date ranges, Live vs Contracted ARR, Trailing Average, Retention Baselines, S&M:Sales Offsets, Expenses Segmentation.” 

That’s one way to think of the BI pendulum. When data teams are popular, generic BI tools are popular, because they give analysts more freedom to define the rules. When data teams fall out of favor, domain-specific tools—which outsource those rules to the product’s designers—start to look more appealing. 

Which, as a pattern of behavior, I guess makes sense. But I’m not sure it’s useful. 

Most of these tools are trying to find the right balance of protection and creativity in how data is modeled and how metrics are defined. The implicit theory here is that companies struggle to turn messy data into consistent and appropriate metrics, and if we can combine the right mix of opinionated best practices with flexible customization, we can all become the data-driven enterprises that we’ve been promised.⁹ 

The problem, however, is that this isn’t what makes successful data-driven companies successful. Amazon isn’t Amazon because it’s great at defining metrics. It’s Amazon because it’s great at interpreting them. Sure, sometimes we miscount how many pizzas we sold, or conflate gross margin with contribution margin. But these aren’t the places where most of us need the most help. We’ve spent thirty years building tools to help us answer our questions. The much harder thing is knowing what to do with those answers once we have them. 

Opinionated business interpretation

A couple weeks ago, Cedric Chin and Sam Taylor launched a new tool, called Xmrit, for creating XmR charts. We talked about these charts a few months ago: They’re a type of time-series chart that uses a handful of simple heuristics to tell you when up-and-down variations are worth paying attention to. It’s a data visualization that tells you how to interpret its results.

In other words, Xmrit is a BI tool that protects you from misunderstanding data. There is no restriction on the data that goes into the tool—you can copy and paste any numbers that you want. But once those numbers are in Xmrit, it has strong opinions about how you should interpret them.

Most BI tools are the inverse. They govern inputs, and provide no guidance on how to make sense of the outputs. And so we screw it up. We screw it up because data interpretation is hard; because the line wiggles; because most analysis is just squinting and directional vibes. 

Xmrit and XmR charts are one example of what a new approach could look like. There are others:

• Metric trees. Metric trees are a structured set of business metrics that are intended to represent how a company actually functions. A revenue metric tree, for example, might break revenue into subcomponents like number of purchases and average revenue per purchase; the number of purchases might then get split into new customer purchases and repeat customer purchases; and so on. Interpretation is baked into metric trees: When a metric changes, walk down the tree to figure out why it changed.

• The one-page BI tool. A few years ago, Dave Kellogg published the one Excel worksheet that he recommends startups use to monitor their sales pipeline. It’s a dense table of specific metrics presented over specific time periods. This sort of tightly-defined BI tool—in which the structure and content is not up for debate—forces people to compare metrics in a particular way, and not prevents them wandering aimlessly through different filters and drilldowns. 

• ARIA, GEM, etc. ARIA is a framework for measuring and improving product adoption. GEM is a framework for describing how companies grow and make money. Though both of them are built on top of common-used metrics, they’re valuable because they tell people how to make sense of those metrics. They are frameworks for metric interpretation, not metric definition.

Just as tools like Linear are unafraid to tell us how we should build software, BI tools could be bolder in telling us how to think about our business. Tell us how to interpret a time-series chart; force us to think about decomposing our business into its important components; give us a rigid summary table of our business that encourages us to debate what we do about the numbers on the page and now how we graph them; impose frameworks on us, and make us choose one.

Of course, not every model would be right for every company; Linear’s probably not right for everyone either. But that seems like a better way to buy software—choose it for its strongest opinions, not for its generic features. 

If I had to guess, all of this is the real reason why BI tools are a tar pit:¹⁰ They solve the wrong problem. They’re focused on protecting us from bad inputs, and don’t do nearly enough to stop us from screwing up the outputs. And until one does that, we’ll keep feeling frustrated, keep churning through tools, and keep wondering why nothing seems to work. 

Maybe it’s time to try another approach. Stop giving us more exploratory interfaces to get lost in. Stop waiting for us to become “data literate.” Assume we aren’t, and won’t be for a long time. If we were jaded about that, what would we build then?¹¹

1. Yes, this is gendered, but ten years ago (and today, but ten years ago too), tech executives were almost all men. Back then, if you were a jaded forty-something-year-old tech female executive, you were probably Sheryl Sandberg, Marissa Mayer, or Meg Whitman. 

2. Florida? Florida!!! I need to forget, so take me to Florida! I've got some regrets, I'll bury them in Florida! What a crush, what a rush, fuck me up, Florida!

3. Hard, it turns out. Your corporate rebrand ruins each car wash’s quirky charm; customers are upset that the owner Cheryl got replaced by a carpetbagger, and that shop attendant Donna got replaced with an iPad. You eventually realize that you don’t know anything about car washes and that Cheryl actually ran a very tight ship. You end up selling all the car washes to a real private equity firm at a loss, and become the CRO of an enterprise AI security company.

4. You never start the venture studio. There were lots of good meetings. We all got excited. The whole thing we got excited about is never going to happen. 

5. Ok, fine, this one might actually be kind of fun. 

6. Another way to lie with statistics: Make a point and confidently back it up with some mostly irrelevant numbers.

7. This blog is 25 percent Griff hype (speaking of, a new album! Announced yesterday! Out on July 19!), 35 percent overwritten ledes about venture capitalists, and 40 percent ideas that I stole from Nan.

8. Or if you do “not currently, and may never, collect, monitor or report certain key operating metrics used by companies in similar industries” and your management and board “does not anticipate relying on any particular key performance metric to make business or operating decisions,” your BI tool won’t care either. 

9. Omni, a new BI tool, makes this argument pretty explicitly. They have a Venn diagram on their homepage that positions Omni as being a combination of being flexible and governed; they say they’re the “only business intelligence platform that combines the consistency of a shared data model with the speed and freedom of SQL.”

10. Well, that and data visualization black hole.

11. I mean, we’d probably a startup studio for incubating AI companies.