The best decision is one

Optionality kills. And why data teams aren’t here to help us make better decisions.

Startup Roadmap (Live Test)

In the course of talking about data startups on the internet, people on the internet occasionally ask me for my thoughts on their data startups. Most of the time, the questions are from founders who are just starting out—they have no employees, a couple rough ideas of what they could to build, and a big decision to make about which one to choose.

Until recently, I’ve tried to answer their questions by doing my best impression of a good analyst. I’d weigh their ideas against each other, try to assess the likelihood that a small company could build each product and make enough noise in crowded market to pick up early customers, match that with my own guesses about the trends in the industry, and, once all the numbers were totaled, read them the printout from my adding machine. I’d then say that I’m just one person; take my thoughts with a healthy grain of salt; keep doing your research; talk to prospective customers; listen to other voices in the community. Until you have more feedback, more evidence, and more conviction, keep your options open. 

With apologies to anyone I’ve had this conversation with…this was bad advice.

Instead, I should’ve said one thing: It doesn’t matter what you choose. If you’ve already done some research and narrowed a dozen mixed ideas down to a couple promising ones, both are probably good. The work you have left to do isn’t more research; it’s to make—and commit, fully and truly—to a decision. 

To propose an analytically heretical axiom about big choices like these, making the decision is more important than the decision you make. The success of a startup—or of a strategic decision in an established company, or even, to go all Chicken Soup for the Soul for a moment,¹ of decisions in life and love—doesn't depend on the specific choice. It depends on how committed you are to that choice. Often, we don’t fail because we choose the wrong thing; we fail because we refuse to choose anything at all.

The euphemisms of indecision

There's nothing terribly interesting about saying it’s good to be decisive. The world is full of quotable lines that make this point—fortune favors the bold;² if you fail, fail while daring greatly; disagree and commit; it’s a leap of faith, Indy. If anything, startups publicly overstate their confidence in what they’re doing, and why it’s destined to succeed.

But there’s a difference between the bold claims in pitch decks and the corporate aphorisms that purportedly define our management style, and how many startups are actually run. In practice, a lot of companies are much more tentative—they just hide it behind euphemisms about iteration, optionality and weakly held opinions.

Consider, to use the example from earlier, an early stage startup looking to decide which product to build. 

The company starts by doing their market research, as the gospel instructs them to. They talk to industry experts, identify key competitors, find unsolved problems, and confirm those pain points with potential customers. They begin to build a product and develop a market position that addresses all of these needs. 

Wisely, they're measured in their commitments. We've only talked to a tiny corner of the market, they say, and our sample is badly biased. It would be foolish to extrapolate a ten-year vision from a couple dozen interviews and feedback from our immediate network.

As they launch their initial private alphas and open betas, the company stays closely engaged with their initial customers. They bless these relationships by christening their first customers as “design partners”. Their react to their users’ reactions: Some features unexpectedly resonate and they double-down on them; others matter less than anticipated, and get trimmed from the roadmap. Early customers nudge the company closer to product market fit.

Some questions, however, remain unanswered. Should the company build for this adjacent persona? Should they try to solve that related problem? We’ll figure this out over time, the startup says. We’ll keep our options open. As we grow and gather more feedback, the right choices will become clear. The paradox of choice is a fallacy; our flexibility, responsiveness, and close observation of the data will give us an advantage over large but rigid incumbents. 

This strategy sounds solid, and it matches what most startup manuals will tell you to do. But it’s often be a bad one, because it can get overwhelmed by another, more powerful force: The erratic and enticing pull of the market.

Early stage startups are often hungry for customers, and, out of a genuine and noble desire to solve problems for people, eager to help people who see potential in what they’re building. When you’re looking for whatever traction you can find, it’s difficult to turn down excited users, and even harder to not be swayed by their opinions.

This introduces a dangerous dynamic. Most customers don’t buy a startup’s product for what it is; they buy it for what they hope it will become. Unless the startup clearly and consistently tells their customers exactly where they’re headed—which is antithetical to agility and informed iteration—customers will do that extrapolation themselves. They’ll see features one and two, and draw a line to features three through ten. For some customers, this roadmap will be an educated guess as to where the startup is headed; for others, it’ll be an unconsciously biased projection in the direction that they want the product to evolve; and for the shrewd ones, it’ll be the feature set they intend to actively pressure the company to build.

Regardless of their motivations, customers will not only tug in slightly different directions, but their continued use of the product will also be, to some degree, contingent on the startup responding to their pull. None of this is said so explicitly though; it happens subtly and slowly, one roadmap chat and prioritization exercise at a time. At each step, the logical step for the startup is to be receptive to the feedback, to leave the door open for future customers and segments, to stay flexible to a changing market, to maintain optionality. But eventually, the company will find itself stretched across multiple visions, unsure of where it’s supposed to go next and who it’s building for, precariously perched between markets and customers that aren’t nearly as stable as Volvo Dynamic Steering. And unlike Jean-Claude Van Damme, most of us can’t do the splits.

Know thyself

To save ourselves from being quartered by the market, we have to take choices off the table. We have to remove optionality. To warp another line from Amazon’s corporate handbook, we have to artificially turn two-way doors into one-way doors.

The problem with maintaining choice is that it allows us to drift between decisions. For the startup that “keeps its options open,” early setbacks might encourage it to experiment with other ideas. They might divert time and money towards the second idea or architect their product in ways that make a potential pivot easier, undercutting their efforts to make the first idea successful.³ 

But more often than not, either idea would work, so long as the company is sufficiently committed to figuring out how to make it work. That’s because most business decisions aren’t like picking a number on a roulette wheel, where we lay down our bet, and wait for the fickle winds of fate to tell us how we did. Instead, outcomes depend on both what we choose and what we do after we choose it. 

Loath as I am to admit it, people like Elon Musk are effective executives for exactly this reason. For all their downsides, megalomaniacs can have one advantage over more grounded leaders: They’re unwilling to even entertain alternative points of view.⁴ Once they’ve made up their mind, their egos refuse to allow them to turn around.⁵ The companies and employees under them have one choice: Make the decision work. 

To be clear, I’m not advocating for all startups to be run like Tesla, or for CEOs to model their leadership style after that of Elon Musk.⁶ And committing to a decision doesn’t have to create the “fight or die” pressure that the original boat burner (and the CEO of Costco) tried to engineer. My point is less dramatic: There are lots of paths to success, but the paths are often long. Once you choose one, stay on it. Stay firm in your identity to both yourself and your customers. And worry about overcoming the particular obstacles on the road you’ve chosen, rather than wondering if the other paths are shorter or less challenging.

Data teams aren’t here for decisions

Data teams have a role to play in this effort. If you ask an experienced analyst what their job is, the enlightened answer is that it’s to help companies make better decisions. I’ve said this plenty of versions of this myself, and when I ask questions like this to our job candidates, it’s the answer I like to hear. 

I’m coming to realize, however, that it’s probably wrong. As data folk, we’re not here to help organizations make better decisions; we’re here to create better results. While that distinction can seem pedantic—isn’t a good decision one that leads to a good result, and a good result the outcome of a good decision?—there is a meaningful difference. 

As mentioned earlier, good outcomes come from both good decisions and a commitment to see that decision through. Data is often helpful for the former requirement. It can be much more problematic for the latter.  

If the data tells a clear story, it can corral diverging opinions into a clear consensus. For the startup debating their product vision, a conclusive piece of analysis can constantly remind people, through the ups and downs of building that roadmap, why it’s still the right one.

But in the more common case, data fails to tell such a neat story. It usually paints a more nuanced picture, full of uncertainty, probabilities, and estimates of “epistemic status.” There will always be reasons why some strategy might be wrong, or why the road not taken might’ve been right. Data that highlights this creates a lot of room for doubt in decisions—doubt that tends to linger long after they’ve been made. In this light, careful, unbiased analysis can flip from an invaluable asset to a subversive liability the moment a company transitions from evaluating their options to executing on them.

As analysts, it can be tempting to say this isn’t our problem. Our job is to figure out what’s true. If those truths require us to reevaluate our prior choices, so be it.

I want to say this. I want this to be my job, to stand on the principle that something roughly shaped like the truth is out there, and to leave the politics of that truth to those who have the courage to make decisions with it. 

But I think that’s a dereliction of our duty. Pharmaceutical companies can’t take credit for the therapeutic wonders of their drugs without taking responsibility for how they can be abused. And we can’t turn a blind eye the effect that our caveats and analytical exceptions can have on a company’s commitment to its prior decisions. Our job—like everyone else’s—is to make things better. It’s to improve results. And we can’t do that just by helping people make decisions. We also have to help people stick to them. 

1. Gotta love that the Chicken Soup for the Soul franchise is owned by a (publicly traded?) parent company called Chicken Soup for the Soul Entertainment, which really gives away the grift on what these supposed inspirational self-help brands are really all about. (Also, if there was ever a time for blithe, uncomplicated stories about how it’s all going to work out, you’d think it’d be in the middle of a stock market crash. Guess not.)

2. No word yet on if fortune favors sleazy confidence men for borderline-criminal pyramid schemes that bankrupt people.

3. This is true in plenty of other domains as well. In writing this blog post, for example, I was waffling between it and a piece on the market’s recent swan dive. Up until 24 hours ago, my mind constantly wandered to that piece, and this post is worse for that indecision.

4. issuing a preemptive correction on this post, regarding megalomaniacs. you do not, under any circumstances, "gotta hand it to them."

5. Falling stock prices, on the other hand, seem to be more persuasive.

6. There’s also a potential logical problem with drawing any lessons from people like Elon Musk. In many competitive environments, the biggest winners are often those who operate with reckless abandon. But that’s different than saying that those who operate with reckless abandon often win. In Silicon Valley, we see the successes like Elon Musk, but rarely hear about the failures. So before we celebrate Elon Musk’s managerial style, we have to consider how many other people tried it and didn’t get so lucky.