The roles of taste and data can co-exist. I'm reminded of this fantastic quote from this NYT article about SSENSE, a fashion aggregator that has become known for its own unique taste:
“Our growth is a result of our having two strong approaches — the art of it and the science of it,” said Krishna Nikhil, the chief merchandising and marketing officer. “We don’t blend art and science. If you blend, you get mush. We toggle.”
Mr. Atallah put it more directly: “I look at data day in and day out. That’s what feeds the intuition. Intuition is not just ‘I feel like doing it.’”
This mode of computer-science thinking, as Eric Hu, Ssense’s design director from 2016 to 2018, said, guides every aspect of the company’s decision making. “Rami has a very clear-cut vision for how things should be,” he said. “And he’s able to make very bold decisions because he requires data and actual intel to prove his hunches.”
Oh nice, this was an interesting read. I like the toggles point, where these things can be sort of different sides of the brain, and different modes of operating applied in different ways, rather than "we need to throw out the designers and just use data to make traditionally taste-based decisions."
“I am sorry, but if you are telling people that you can rank the effectiveness of a political ad “down to a tenth of a percentage point of precision in multiple categories,” you aren’t learning something; you are selling something.” That quote is from the journalists, not the people doing the ranking. The folks doing the ranking are serious social scientists who actually know how to calculate standard errors…
It depends what the standard error is. My point is that I guarantee they calculate that and put it in decks and memos but that the journalists were ignoring that. And sure—you can always question methods—external validity of RCT’s etc. But the original snark was at falsely claimed precision, which I guarantee the practitioners are not doing.
Sure, you can calculate standard errors, but is the data *really* telling you that an ad was 34.6% effective? Not 33.0%? At the bottom of all this precision is the assumption that 'effectiveness' can even be measured and that the methods for measuring and collecting the data are remotely as precise as the math.
My take is that we can over rely on data and think the precision we've calculated is meaningful when it's not. Just because I can give you a result of 34.6%, doesn't mean we're being data driven, in the sense that we're working with reliable and accurate measurements. If data driven simply means we're using numbers from some sort of measurement, that's not necessarily anything to crow about.
Yeah, I get that that quote is from the Times and not necessarily (though not necessarily *not*) the firm themselves, and I'm sure there are plenty of people in that org that know that precision is sort of fake. But you can know it's fake and still sell it. And I suspect they are selling it.
No, of course it's not a fad. Business has always and will always whatever data is useful and obtainable in decision making. Even in fashion. But this certainly doesn't mean every decision is going to be based on a dashboard in Tableau derived from set of tables in Snowflake.
As for stocks - no amount of data will *ever* allow you to reliably and consistently predict the stock price 24 hours into the future for anything. Not unless humans are completely out of the loop. Completely. But then who cares?
Like, sure, but isn't that basically saying "nothing changes and nothing has ever changed?" I agree that there will always be degrees of data; there will always be some desire to be data-driven; stock prices will never be perfectly predictable; etc. But it seems pretty clear that, over the last 20-30 years, we've tried to introduce data into a lot more decisions than we did before. My question is mostly about that. The rough assumption seems to be that it's a one-way ratchet. But could it not go in reverse?
"But it seems pretty clear that, over the last 20-30 years, we've tried to introduce data into a lot more decisions than we did before."
Yes, definitely, because we've been able to more easily capture more data about people and processes. Whether or not it has all been useful is debatable. Yes, I think some of this will go in reverse because it's not all useful. Witness the debate/growing dismissal of marketing attribution data.
I really doubt all the data collected in an effort to predict stock prices is useful, but the payoffs are so high when someone momentarily hits that people will keep trying.
Exactly, I think that's my question. There was a lot of hype around the big explosion of stuff in the last 20-30 years, and part of that hype was selling that as this inevitable march upwards. And to your point about the dismissal of marketing attribution, that's a good example of "the vibes have shifted" thing I was trying to say: It seems like there's now a lot more skepticism of that stuff - and to some degree, interest in other stuff - than there was a few years ago.
(On stock prediction, yeah, the point of whole stock thing wasn't really to say that we can *predict* stock prices. It was just that there are philosophical regime changes. For a long time, there has been - and still is, really - a regime that says stocks should be roughly valued based on these various financial metrics. Matt Levine's point was perhaps that regime will recede, even though going back to vibes and animal spirits seems like a regression. And my point is that maybe the "data driven" regime recedes to something else, even if that also feels like a regression.)
On stocks, I think in the long run they are valued primarily on the earnings the company produces. In the short run, *anything* can drive the price. And I think this has become a lot more volatile with the rise of easy retail investing. But I think vibe investing has been around since stocks have been traded, too. There's just more of it and it's more obvious.
That's the question though, isn't it? In the long run, *are* they valued on earnings? People have been expecting GameStop to go back to "normal" for years. Tesla has basically been a meme stock for half a decade. If people like GME because it's a fun collectible, there's nothing that says it can't just that way forever.
But that's sort of my point about this being so hard to accept. It's really tempting to believe that this stuff has to have some "long run" value that is "correct" - or, maybe more precisely, really hard to believe that it "long run" value isn't based on anything. But it doesn't have to be! Most things *aren't;* they're worth is mostly a function of hype and popularity and how socially desirable something is. Why should stocks *not* behave that way too?
I have been thinking about this a lot recently. I’m thinking more now about data-centric - companies who build around data vs data driven. The people are in the drivers seat - but some companies build around data - and others around vibes or founders or LinkedIn audiences. I think sometimes it’s a philosophy and not necessarily the right one - or the only right one… and I agree it depends on the industry and problem you are solving.
Yeah, I see it sort of like "customer obsession." There are a lot of companies that claim to be customer-obsessed, and will have CEOs that march around talking about how everyone needs to be customer obsessed. Which, sure, that's probably a good way to be. But I don't think you can just say it and go through the motions of it; it's got to be baked in from the beginning. That's sort of the point of obsessions - you start with them first, and it becomes the identity, not the other way around. But lots of people try to do it in reverse.
If you bake the data driven thing in from the beginning, I think it can work, not because data has all the answers, but because it can be a unifying operational guidepost. But like customer obsession, I think that only works if it's the natural DNA of the company, and not some painted on gloss that the CEO wishes were true.
yes - the ironic one here is I think at one point you could say Amazon was both customer obsessed and data-driven. However the data-driven was in service to the customer obsession.
While that strange truth company of TMTG may seem built on thin ice, we have to also understand that the stick that is used for its measurement, the dollar is on thin ice: trust only.
Indeed, more and more companies now use data intensively. It seems a necessity and we risk making the data so important that expenditure on that outdoes any possible revenue. Do we use the data we have (creatively) or are we hoarding unnecessary data just because it could offer value?
Do we use the data we have wisely throwing AI methods at them or should we just have a good visualisation to look at and a simple statistic or two?
If you can't create a good algorithm out of your AI work, was your AI worth it?
I do think there's something to the "use it more creatively" thing, where a lot of the potential uses comes from stretching it. Half the value to me is in very mechanical, basic reporting, and half the value is in trying wild stuff.
Claiming to be data-driven seems to be a fad. There are certain personality types that are more data-driven than others. These people are at the cutting edge of science, tech, finance, et al. All of these organizations and disciplines were data-driven long before it became fashionable to claim to be so, and will remain data-driven even after all the CDOs of the world get fired for being useless.
Interestingly, the truly data-driven orgs have a tell: they don't have CDOs.
I think your outliers can be very easily explained by psychology, and more specifically, psychopathy. The allure of psychopathic traits to the often "volatile" or "exciting" business and political world drive people to take risks they normally do not. I don't think data is a fad at all, but rather, it matters what sort of data you collect for stable vs unstable situations.
I was tickled to see you ruminating on the history of analytics! I have for a while been noodling on trying to explore that topic more deeply. Moneyball is certainly one branch. I once ran across a discussion from intelligence analysts that felt awfully resonant with my professional experience. Data quality concerns, the futility of writing the perfect memo that gets ignored by leaders, getting held accountable for bad outcomes when all you said was your estimate was that success was more likely than not, and so on. Maybe there's something about "staff officers" in here too; the cadre of people who support a senior leader in their thinking and operations. Feels similar to some elements of data leadership.
My not-yet-researched sense is that "analyst" as a title was an intelligence or defense title before it was a private sector financial title. Perhaps this is part of the etymology of "business intelligence?" Again, zero evidence of this but I have a hunch there's something going on in the history that would be illuminating.
That'd be a really interesting story. It seems like most people's default (including mine) is that the whole analyst thing is mostly from financial and business analysts - because they use data, I guess - but this makes me wonder if it's more of a child of intelligence analysts. They might less "data" people who live in spreadsheets or whatever, but to your point, there's a style of work there that might be much better match than an accountant.
Either way though, I do hope someone writes a story about the arc of all of this someday. The data science tangent, the analytics engineering thing - there are so many trends and things that got popular and faded and got rebranded. There's gotta be a definitive history of all of this somewhere.
It never went away. Ask probably any executive. I can't think of many management roles where some intuition isn't part of decision making because we can't measure everything.
I think I agree with both of these points? "Founder Mode" or whatever never went away, but people had less permission to operate that way (partly because the push was for execs to be data driven). The shift to me is that the founder mode style is becoming acceptable, and the data driven style is feeling more dated. (Which isn't necessarily a statement about what's good, or what should be, or any of that. It's just a statement about what's in style.)
"The shift to me is that the founder mode style is becoming acceptable, and the data driven style is feeling more dated."
I interpreted Founder Mode as ensuring that you're in on the key decisions, rather than handing them off to other people. Brian Chesky gave the talk, and he runs a very "data-driven" company (they gave us Airflow!)
Where are you seeing this applied and successful in reality (versus Venture Capitalist marketing)?
I'm not saying that founder mode is good or bad, or that I've seen it work. I do think decisiveness works, because I think relentlessly going after a somewhat imperfect thing is better than over optimizing the chase a more perfect thing (more on that here: https://benn.substack.com/p/the-best-decision-is-one)
But, in theory, sure, I guess you could be in on more decisions and not delegate, and you yourself be very data-driven. But those things are pretty spiritually different. If you're bulldozing in, making key decisions yourself, you're probably making them based on some personal beliefs and intuitions, not data. That's sort of the point of being data driven - the decision is in the numbers, not in someone's head. And unless the founder has some supernatural ability to read the numbers that other people don't have - which is definitely not what Paul Graham was trying to say - then there shouldn't be that much reason for a founder to need to be involved in key data-driven decisions.
(In practice, I suspect what happens at Airbnb is Brian Chesky looks at numbers, he picks and chooses which ones he pays attention to, he makes a decision, partly based on numbers, but heavily colored by his intuition. Which, is that being "data-driven?" I don't know, though I'd say it's not being data driven in the way people have classically talked about it for the last decade.)
I agree that's probably his process at Airbnb! Though I'd be slightly less...cynical (?) in my description. Data and intuition are complementary, and you don't need to intuit many things you can measure.
Sure, I think that's fair. I guess the more precise way of putting all of this is, there's always been some balance between data and intuition; over the last 20 years, that balance has tilted towards data; now, it feels like balance is moving the other way.
What a strange take. There's no dichotomy between "gut" and "data". There's no magic business instinct. The idea the Elon Musk and the engineers and SpaceX and Tesla are just winging it is silly. One of the highly publicized reasons for making cuts at Twitter was the low revenue per employee figure.
Oh well. Can't wait for a bunch of people to start saying "who needs data?" so those of us who understand why data is useful in decision making can get a huge edge...again.
I think I would frame the question pretty differently. I'm certainly not saying engineers at SpaceX are winging it, nor am I saying Elon Musk did what he did at Twitter without looking at some basic charts. But, I do think it's pretty clear he approached it much more aggressively, more quickly, and bluntly than a bunch of very "data-driven" McKinsey consultants would've. To your point, he probably saw a handful of numbers, said "wow that employee number is high and that revenue number is low" and then hacked off most of the employees. And that's sort of the point - that's both 1) *not* something people would've called data driven two year ago, and 2) seems to be an increasingly more popular way to do things today.
My point is that for the last 15 years, people have written thousands of articles like these, about how companies need to be more data driven, how it's hard, how it's critical, how it's a huge competitive advantage, and so on. These articles - and the industry around them - aren't arguing for looking at basic charts; they're saying you need to be much more rigorous, detailed, and analytical. So while we could call looking at basic charts and dashboards and revenue numbers data driven, I think that's a pretty big deviation from how the term has been used (and sold) for a while.
Put differently, were sports teams in the 1990s data driven? They looked at stats. But they did so in a way that was far more rudimentary than teams do today. My question is, is that latter thing a fad?
Ah. Yes, definitely on how it's been sold; in some ways 'data driven' has been pushed like magic beans by the data industry. My definition is not nearly as demanding.
Yes, I think it's possible to over rely on numbers and data . We can trick ourselves into thinking we can precisely measure something when we cannot.
I think they are arguing to look at basic charts, because that's where you start. I'm not sure about your experience, but mine is that 90% of companies aren't even doing that. You start simple and go from there. You can't jump to the end and magically be Google.
Yes, 1990s sports teams were data-driven. It was rudimentary because those were the tools, data and skills available. Now it is more advanced because the tools, data and skills have advanced. I think this is the point.
Maybe our difference here is being data driven versus "data driven" (the brand)?
Yeah, I'm definitely talking about the brand, though I think it's hard to fully separate the two. I'd argue that people went to the effort to be data driven because the brand was popular. If the brand becomes unpopular, then a lot fewer people will do the thing and will do a new thing instead.
(I do think I'm somewhat skeptical of the thing too, or at least the thing as it was promised by the brand. I'm with you on the basics, and think that's really valuable. But I'd argue that the brand said that there was a lot of stuff beyond the basics that was also valuable, and I don't know if that stuff is.)
The roles of taste and data can co-exist. I'm reminded of this fantastic quote from this NYT article about SSENSE, a fashion aggregator that has become known for its own unique taste:
https://www.nytimes.com/2021/11/23/style/the-sensibility-of-ssense.html
“Our growth is a result of our having two strong approaches — the art of it and the science of it,” said Krishna Nikhil, the chief merchandising and marketing officer. “We don’t blend art and science. If you blend, you get mush. We toggle.”
Mr. Atallah put it more directly: “I look at data day in and day out. That’s what feeds the intuition. Intuition is not just ‘I feel like doing it.’”
This mode of computer-science thinking, as Eric Hu, Ssense’s design director from 2016 to 2018, said, guides every aspect of the company’s decision making. “Rami has a very clear-cut vision for how things should be,” he said. “And he’s able to make very bold decisions because he requires data and actual intel to prove his hunches.”
Oh nice, this was an interesting read. I like the toggles point, where these things can be sort of different sides of the brain, and different modes of operating applied in different ways, rather than "we need to throw out the designers and just use data to make traditionally taste-based decisions."
“I am sorry, but if you are telling people that you can rank the effectiveness of a political ad “down to a tenth of a percentage point of precision in multiple categories,” you aren’t learning something; you are selling something.” That quote is from the journalists, not the people doing the ranking. The folks doing the ranking are serious social scientists who actually know how to calculate standard errors…
It depends what the standard error is. My point is that I guarantee they calculate that and put it in decks and memos but that the journalists were ignoring that. And sure—you can always question methods—external validity of RCT’s etc. But the original snark was at falsely claimed precision, which I guarantee the practitioners are not doing.
Sure, you can calculate standard errors, but is the data *really* telling you that an ad was 34.6% effective? Not 33.0%? At the bottom of all this precision is the assumption that 'effectiveness' can even be measured and that the methods for measuring and collecting the data are remotely as precise as the math.
What does any of this have to do with being data-driven, as opposed to a lack of statistical knowledge?
My take is that we can over rely on data and think the precision we've calculated is meaningful when it's not. Just because I can give you a result of 34.6%, doesn't mean we're being data driven, in the sense that we're working with reliable and accurate measurements. If data driven simply means we're using numbers from some sort of measurement, that's not necessarily anything to crow about.
Yeah, I get that that quote is from the Times and not necessarily (though not necessarily *not*) the firm themselves, and I'm sure there are plenty of people in that org that know that precision is sort of fake. But you can know it's fake and still sell it. And I suspect they are selling it.
No, of course it's not a fad. Business has always and will always whatever data is useful and obtainable in decision making. Even in fashion. But this certainly doesn't mean every decision is going to be based on a dashboard in Tableau derived from set of tables in Snowflake.
As for stocks - no amount of data will *ever* allow you to reliably and consistently predict the stock price 24 hours into the future for anything. Not unless humans are completely out of the loop. Completely. But then who cares?
Like, sure, but isn't that basically saying "nothing changes and nothing has ever changed?" I agree that there will always be degrees of data; there will always be some desire to be data-driven; stock prices will never be perfectly predictable; etc. But it seems pretty clear that, over the last 20-30 years, we've tried to introduce data into a lot more decisions than we did before. My question is mostly about that. The rough assumption seems to be that it's a one-way ratchet. But could it not go in reverse?
"But it seems pretty clear that, over the last 20-30 years, we've tried to introduce data into a lot more decisions than we did before."
Yes, definitely, because we've been able to more easily capture more data about people and processes. Whether or not it has all been useful is debatable. Yes, I think some of this will go in reverse because it's not all useful. Witness the debate/growing dismissal of marketing attribution data.
I really doubt all the data collected in an effort to predict stock prices is useful, but the payoffs are so high when someone momentarily hits that people will keep trying.
Exactly, I think that's my question. There was a lot of hype around the big explosion of stuff in the last 20-30 years, and part of that hype was selling that as this inevitable march upwards. And to your point about the dismissal of marketing attribution, that's a good example of "the vibes have shifted" thing I was trying to say: It seems like there's now a lot more skepticism of that stuff - and to some degree, interest in other stuff - than there was a few years ago.
(On stock prediction, yeah, the point of whole stock thing wasn't really to say that we can *predict* stock prices. It was just that there are philosophical regime changes. For a long time, there has been - and still is, really - a regime that says stocks should be roughly valued based on these various financial metrics. Matt Levine's point was perhaps that regime will recede, even though going back to vibes and animal spirits seems like a regression. And my point is that maybe the "data driven" regime recedes to something else, even if that also feels like a regression.)
On stocks, I think in the long run they are valued primarily on the earnings the company produces. In the short run, *anything* can drive the price. And I think this has become a lot more volatile with the rise of easy retail investing. But I think vibe investing has been around since stocks have been traded, too. There's just more of it and it's more obvious.
That's the question though, isn't it? In the long run, *are* they valued on earnings? People have been expecting GameStop to go back to "normal" for years. Tesla has basically been a meme stock for half a decade. If people like GME because it's a fun collectible, there's nothing that says it can't just that way forever.
But that's sort of my point about this being so hard to accept. It's really tempting to believe that this stuff has to have some "long run" value that is "correct" - or, maybe more precisely, really hard to believe that it "long run" value isn't based on anything. But it doesn't have to be! Most things *aren't;* they're worth is mostly a function of hype and popularity and how socially desirable something is. Why should stocks *not* behave that way too?
"It seems like there's now a lot more skepticism of that stuff"
I think the stuff you're talking about here is marketing, not data.
Case in point for dubious business valuations: NFTs.
Definitely an "animal spirit" thing.
But ultimately, the value of things is based on how much people are willing to pay for them.
Yeah, I mean, if people will buy a jpeg for a gazillion dollars, who are the rest of us to say it's not worth a gazillion dollars?
I have been thinking about this a lot recently. I’m thinking more now about data-centric - companies who build around data vs data driven. The people are in the drivers seat - but some companies build around data - and others around vibes or founders or LinkedIn audiences. I think sometimes it’s a philosophy and not necessarily the right one - or the only right one… and I agree it depends on the industry and problem you are solving.
Yeah, I see it sort of like "customer obsession." There are a lot of companies that claim to be customer-obsessed, and will have CEOs that march around talking about how everyone needs to be customer obsessed. Which, sure, that's probably a good way to be. But I don't think you can just say it and go through the motions of it; it's got to be baked in from the beginning. That's sort of the point of obsessions - you start with them first, and it becomes the identity, not the other way around. But lots of people try to do it in reverse.
If you bake the data driven thing in from the beginning, I think it can work, not because data has all the answers, but because it can be a unifying operational guidepost. But like customer obsession, I think that only works if it's the natural DNA of the company, and not some painted on gloss that the CEO wishes were true.
yes - the ironic one here is I think at one point you could say Amazon was both customer obsessed and data-driven. However the data-driven was in service to the customer obsession.
True, though that might be why they're like the fifth most valuable company of all time or whatever. If you pull all both, you win the lottery.
Yup
While that strange truth company of TMTG may seem built on thin ice, we have to also understand that the stick that is used for its measurement, the dollar is on thin ice: trust only.
Indeed, more and more companies now use data intensively. It seems a necessity and we risk making the data so important that expenditure on that outdoes any possible revenue. Do we use the data we have (creatively) or are we hoarding unnecessary data just because it could offer value?
Do we use the data we have wisely throwing AI methods at them or should we just have a good visualisation to look at and a simple statistic or two?
If you can't create a good algorithm out of your AI work, was your AI worth it?
Rethink instead of recalculate?
I do think there's something to the "use it more creatively" thing, where a lot of the potential uses comes from stretching it. Half the value to me is in very mechanical, basic reporting, and half the value is in trying wild stuff.
Claiming to be data-driven seems to be a fad. There are certain personality types that are more data-driven than others. These people are at the cutting edge of science, tech, finance, et al. All of these organizations and disciplines were data-driven long before it became fashionable to claim to be so, and will remain data-driven even after all the CDOs of the world get fired for being useless.
Interestingly, the truly data-driven orgs have a tell: they don't have CDOs.
That point about CDOs...does not seem wrong.
I think your outliers can be very easily explained by psychology, and more specifically, psychopathy. The allure of psychopathic traits to the often "volatile" or "exciting" business and political world drive people to take risks they normally do not. I don't think data is a fad at all, but rather, it matters what sort of data you collect for stable vs unstable situations.
I'm not sure which outliers you're referring to?
I was tickled to see you ruminating on the history of analytics! I have for a while been noodling on trying to explore that topic more deeply. Moneyball is certainly one branch. I once ran across a discussion from intelligence analysts that felt awfully resonant with my professional experience. Data quality concerns, the futility of writing the perfect memo that gets ignored by leaders, getting held accountable for bad outcomes when all you said was your estimate was that success was more likely than not, and so on. Maybe there's something about "staff officers" in here too; the cadre of people who support a senior leader in their thinking and operations. Feels similar to some elements of data leadership.
My not-yet-researched sense is that "analyst" as a title was an intelligence or defense title before it was a private sector financial title. Perhaps this is part of the etymology of "business intelligence?" Again, zero evidence of this but I have a hunch there's something going on in the history that would be illuminating.
That'd be a really interesting story. It seems like most people's default (including mine) is that the whole analyst thing is mostly from financial and business analysts - because they use data, I guess - but this makes me wonder if it's more of a child of intelligence analysts. They might less "data" people who live in spreadsheets or whatever, but to your point, there's a style of work there that might be much better match than an accountant.
Either way though, I do hope someone writes a story about the arc of all of this someday. The data science tangent, the analytics engineering thing - there are so many trends and things that got popular and faded and got rebranded. There's gotta be a definitive history of all of this somewhere.
The main insight from posts like yours and Founder Mode is that intuition, once deemed unprofessional, has re-emerged as a valid decision-making tool.
It never went away. Ask probably any executive. I can't think of many management roles where some intuition isn't part of decision making because we can't measure everything.
I think I agree with both of these points? "Founder Mode" or whatever never went away, but people had less permission to operate that way (partly because the push was for execs to be data driven). The shift to me is that the founder mode style is becoming acceptable, and the data driven style is feeling more dated. (Which isn't necessarily a statement about what's good, or what should be, or any of that. It's just a statement about what's in style.)
"The shift to me is that the founder mode style is becoming acceptable, and the data driven style is feeling more dated."
I interpreted Founder Mode as ensuring that you're in on the key decisions, rather than handing them off to other people. Brian Chesky gave the talk, and he runs a very "data-driven" company (they gave us Airflow!)
Where are you seeing this applied and successful in reality (versus Venture Capitalist marketing)?
I'm not saying that founder mode is good or bad, or that I've seen it work. I do think decisiveness works, because I think relentlessly going after a somewhat imperfect thing is better than over optimizing the chase a more perfect thing (more on that here: https://benn.substack.com/p/the-best-decision-is-one)
But, in theory, sure, I guess you could be in on more decisions and not delegate, and you yourself be very data-driven. But those things are pretty spiritually different. If you're bulldozing in, making key decisions yourself, you're probably making them based on some personal beliefs and intuitions, not data. That's sort of the point of being data driven - the decision is in the numbers, not in someone's head. And unless the founder has some supernatural ability to read the numbers that other people don't have - which is definitely not what Paul Graham was trying to say - then there shouldn't be that much reason for a founder to need to be involved in key data-driven decisions.
(In practice, I suspect what happens at Airbnb is Brian Chesky looks at numbers, he picks and chooses which ones he pays attention to, he makes a decision, partly based on numbers, but heavily colored by his intuition. Which, is that being "data-driven?" I don't know, though I'd say it's not being data driven in the way people have classically talked about it for the last decade.)
I agree that's probably his process at Airbnb! Though I'd be slightly less...cynical (?) in my description. Data and intuition are complementary, and you don't need to intuit many things you can measure.
Sure, I think that's fair. I guess the more precise way of putting all of this is, there's always been some balance between data and intuition; over the last 20 years, that balance has tilted towards data; now, it feels like balance is moving the other way.
Of course. The point is to limit where you have to use intuition, and where you can use "hard data", by measuring things.
I mean, we're not taking estimates on how much money we made last year, are we?
I don't think we disagree. If you can use valid and reliable data to make a decision, that's better.
What a strange take. There's no dichotomy between "gut" and "data". There's no magic business instinct. The idea the Elon Musk and the engineers and SpaceX and Tesla are just winging it is silly. One of the highly publicized reasons for making cuts at Twitter was the low revenue per employee figure.
Oh well. Can't wait for a bunch of people to start saying "who needs data?" so those of us who understand why data is useful in decision making can get a huge edge...again.
I think I would frame the question pretty differently. I'm certainly not saying engineers at SpaceX are winging it, nor am I saying Elon Musk did what he did at Twitter without looking at some basic charts. But, I do think it's pretty clear he approached it much more aggressively, more quickly, and bluntly than a bunch of very "data-driven" McKinsey consultants would've. To your point, he probably saw a handful of numbers, said "wow that employee number is high and that revenue number is low" and then hacked off most of the employees. And that's sort of the point - that's both 1) *not* something people would've called data driven two year ago, and 2) seems to be an increasingly more popular way to do things today.
Why wouldn't making decisions based on comparatively poor revenue/employee numbers be data driven?
My point is that for the last 15 years, people have written thousands of articles like these, about how companies need to be more data driven, how it's hard, how it's critical, how it's a huge competitive advantage, and so on. These articles - and the industry around them - aren't arguing for looking at basic charts; they're saying you need to be much more rigorous, detailed, and analytical. So while we could call looking at basic charts and dashboards and revenue numbers data driven, I think that's a pretty big deviation from how the term has been used (and sold) for a while.
- https://hbr.org/2021/02/why-is-it-so-hard-to-become-a-data-driven-company
- https://hbr.org/2021/06/legacy-companies-need-to-become-more-data-driven-fast
- https://hbr.org/2022/02/why-becoming-a-data-driven-organization-is-so-hard
Put differently, were sports teams in the 1990s data driven? They looked at stats. But they did so in a way that was far more rudimentary than teams do today. My question is, is that latter thing a fad?
Ah. Yes, definitely on how it's been sold; in some ways 'data driven' has been pushed like magic beans by the data industry. My definition is not nearly as demanding.
Yes, I think it's possible to over rely on numbers and data . We can trick ourselves into thinking we can precisely measure something when we cannot.
I think they are arguing to look at basic charts, because that's where you start. I'm not sure about your experience, but mine is that 90% of companies aren't even doing that. You start simple and go from there. You can't jump to the end and magically be Google.
Yes, 1990s sports teams were data-driven. It was rudimentary because those were the tools, data and skills available. Now it is more advanced because the tools, data and skills have advanced. I think this is the point.
Maybe our difference here is being data driven versus "data driven" (the brand)?
Yeah, I'm definitely talking about the brand, though I think it's hard to fully separate the two. I'd argue that people went to the effort to be data driven because the brand was popular. If the brand becomes unpopular, then a lot fewer people will do the thing and will do a new thing instead.
(I do think I'm somewhat skeptical of the thing too, or at least the thing as it was promised by the brand. I'm with you on the basics, and think that's really valuable. But I'd argue that the brand said that there was a lot of stuff beyond the basics that was also valuable, and I don't know if that stuff is.)