MDS tools have been solving the same technical problems on their own. Notably, integrations. Data movement, orchestration, transformation, quality, bi tools... all need integrations for each data storage system... And each tool solved it from scratch, with no common ground. Let me explain, when I worked for a BI tool, we wrote integrations to read data from data storages. When I worked for a data movement tool, we wrote connectors to read and write data from data storages. When I worked for a data quality tool, we wrote connectors to execute queries (tests) on data storages. Didn't work for a data orchestration tool, but operators basically make API calls to external systems, notably data storages. The value is in the data, or metadata if the system is not a data storage per se. And the common thing, is making api calls to these systems: read data, write data, query data (execute tests), trigger actions (orchestration). If there was a common framework to interact with external systems, then I think data tool companies could be more ambitious with a clear vision focused around data. We are missing standards, and a vision like you said. A standard to connect to any external system and be able to read, write, query and trigger actions. Then we need user experiences on top of that. And probably also a common standard on what to build user experiences.
Yeah, this has always bothered me too. Everyone builds connectors, or charts, or query interfaces, or whatever else. The whole stack is _sorta_ a bunch of components (a database, ETL, BI, etc), but each of those component is made from smaller, very duplicative components. Every product is like something on the taco bell menu - same ingredients, different arrangement.
I'd even venture in drawing a parallel with mobile apps: there's a lot of overlap between different apps, some even provide exactly the same value statement.
But it's all to the benefit of the marketplaces, not the users unfortunately.
The users have to navigate a sprawling landscape of ever-multiplicating products that all sell slightly differently a part of the same promise...
In our (data practitioners) case, the marketplaces are the (data) cloud providers which only stand to benefit from such a wide array of vendors all implementing something on top of their cloud.
I think there's a big chance we're going to witness exactly the same scenario play out with the GPTs marketplaces.
There seems to be a stronger open source core this time around, but will that be enough?
That's true, though I think there are two very big difference between data tools and apps: 1) Apps are cheap or free, and 2) they don't really need to work together. Data tools cost a lot to make, so they tend to be pretty expensive to buy. And, they all have to work together; you've got to actually integrate them into each other. So while a messy app ecosystem is is annoying, a messy data ecosystem is borderline nonfunctional.
Also, any outsider can look at Matt Turck's Machine learning AI, and data landscape and think - how in the, what?, am I to make sense of this? There's no connections illustrated between all of these products, and even if there was, it would look like every conspiracy theorist's pictures with string mood board. Hard to make sense of any gathering point in the chaos indeed.
I audibly laughed when I got to " Anyway, speaking of the modern data stack." I think a fair amount of us that have been around the block a while can see exactly what you're highlighting so succinctly in this essay. Every conference I wind up going to, I keep thinking, aren't these the same sorts of problems we've been trying to "solve" for the last 20 some odd years I've been doing this, just re-packaged against the latest in the Gartner hype cycle? Having a Schelling point has to be something that is at the front of whatever evolution the modern data stack turns into. In a way, it's that "so what?" we're constantly grappling for in the majority of the solutions we're trying to deliver.
I was recently reading yet another article about how data teams aren't listened to, and how we can convince stakeholders to pay attention to us, and I think I've started to take an even more cynical turn on all of this, to be honest. I'm starting to think the entire approach we've taken to data (not just the tooling, but how we attempt to apply it to businesses) is pretty broken.
Like, If you make a product (in this case, the output of a data team) and nobody wants to buy it, for years, and you have to keep "training people how to use it," and they keep trying to work around you, and this happens, over and over and over again, until people who are selling the product get frustrated in quit...maybe it's the product you're selling?
Two thoughts - I don’t know him personally - but I think Tristan is one of the type 1 founders - and I agree there are not many in the MDS ecosystem. I really respect what dbt has done around mission and ecosystem.
I think a phase II of this would be getting engineers / developers AND data scientists full buy in for the MDS or analytics stack - or whatever we are calling it now. If those groups actually started fully adopting snowflake and dbt and other tools in the space I think there is a huge second wave of growth. (I know some did - but I think a lot did not). Then the promise of a unified data platform with data you trust to run your business on could actually be more true - and that was always my north star goal. Plus the added benefit of all your data in one place to hook up to AI stuff is a bonus - I think…
I think I half buy the second part of that. It seems possible, though I'm increasingly starting to think it's also possible that the entire arrangement - like, analytics, as a thing - doesn't really work. Not never, and that doesn't mean _data_ isn't useful, but I'm not sure that we'll ever get to a point where we have this whole data democratization utopia (or anything even close to it) that we've been talking up for a long time.
Ya - I get that too, I go back and forth. I also wonder if companies will be trying more dual role things - like let’s higher a sales manager and then buy an AI data tool - now we don’t need a sales analyst. I suspect this will happen and think it typically won’t work out well. AI seems like it will encourage armies of generalist - which might become its own problem…
Yeah, I'm not sure that will work either. I think I'm slowly going even further than that though, and wondering if, like, "insights" or whatever are actually there. My question is becoming, is analytics a failure? Not the way we do it, but the whole exercise?
It was this post that did me in. It's a perfectly good post; there's nothing about it specifically that bad or anything. But it's just...the same thing people have been trying to do forever? Like, we're on year 10 of begging our customers to buy our product, of trying to teach them how to use it, of telling them they don't understand how valuable it could be, and all of that. At some point, doesn't it just seem more likely that what we're selling isn't very useful??
Ha, ya - you gotta be open to that possibility - for sure. I actually had a discussion this week with a friend who thinks it’s a generational problem and that he doesn’t think Gen X will make the leap to be more “data driven” - I’m not so convinced that’s the issue. I think part of the issue is selling such flexible un-opinionated tools.
Data Dog is a great analytics tool for example - but it clearly went after a market and build out specific purpose built features for security, troubleshooting errors, monitoring, etc. Maybe thats part of the solution.
Excellent critique. I've seen half a dozen different ways to construct a data architecture, each using a different set of vendors, a different approach to common problems, and a myriad of permutations of the MDS.
I found the use of the Schelling point as a "test" for the viability of a product / idea very compelling. It's telling that the MDS seems to be failing in that area. We must take a step back and reconsider.
I'm of the optimistic bent that data is a thing. That data at the enterprise scale, especially the global / multi-national enterprise scale, is a both a hard thing but a thing of immense value. Knowing the number of pumps in operation in an energy company is a good and necessary thing, and connecting that both to the maintenance records as well as to the revenue projections should be possible, maybe even easy. But for some reason our MDS in its current incarnation has struggled to solve the core challenges of the modern enterprise.
The journey must continue and this is a marker of a critical waypoint.
I appreciate that. And yeah, that's the nagging thing to me, is it sure *feels* like this should be useful? But something about it seems to keep coming up short.
My nagging concern is that these solutions are driven by technologists, based on their slightly too abstract point of view. I wonder what a data stack or data platform would look like that was driven only by a specific domain's business requirements?
My feeling is that the technologists are off just enough in defining the abstractions that it restricts their ability to understand/address the true problem. We technologists tend to be enamored by our patterns and frameworks, as if they are the thing. Too often we hijack the requirements discussion with our technical concerns, crowding out and obfuscating the actual business problems.
That raises a really interesting question to me - why haven't any non-data people started data companies?
Like, I sorta get it; it doesn't feel like your area of expertise, or whatever. But at the same time, it also feels like it's possible that they haven't because they just don't need what it's offering?
I think that they have, but we don't see them as data solutions. They are domain-specific applications with complex data requirements. These are the things we're often making the connectors for. But they tend to be focused on a specific set of problems in the domain, and as a result it is left to the company (or their consultants) to do the integration work with their other silo'd systems.
These types of applications would never pass muster with proper technologists, and tbh they are often pedestrian in their technical architectures and interfaces. But they work despite the lack of innovation (maybe that's a feature?) and can be cash cows.
That's probably true - we don't see them as data companies, so they fly under our radar. I'm not sure if that's a good or bad thing though. On one hand, it could mean there's all this opportunity, if only people combined the utility of those tools with the technical capabilities of tools build by data people. On the other hand, maybe that means the technical capabilities don't really add that much? I don't know.
Definitely saw this in healthcare and insurance...some of the solutions ignored all the hard stuff. Was basically, if you can figure out how to harmonize, cleanse, and master your data into my model, i can spit out some whiz-bang visualizations for you.
The costs are the part that never got better. I've been "On the Cloud" since 2009, and I've been six months away from the true cost savings for the entire time. Until the last few years people were fairly rational about it, but lately the dynamic has become accelerationist, with execs spending millions on "cloud native" snake oil and blundering into layoffs.
DHH's recent Cloud Exit of 37signals' app hosting has been impressive, and I think the cost differentials are even bigger for the data space.
I pretty mixed on this. Though I think it's fair to say that SaaS products have a bunch of hidden and obfuscated costs (which was what this piece was basically about https://benn.substack.com/p/data-and-the-almighty-dollar), that also seems true for self-hosting? Like, to me, the biggest benefit of SaaS isn't necessarily the dollar cost, but the ease of management, and not having to deal with the headaches of self-hosting. Like, I'd probably be willing to pay *more* for SaaS?
That said, there's a danger with the ease of use, which is that it's easy to buy without really thinking that carefully about it. So maybe that actually eats away a lot of the gains.
My experience agrees with his statement that "80-90% of what you need to know to run your own hardware these days is the same as what you need to know to operate in the rental clouds. From containers to load balancers to monitoring to performance analysis, and a million other topics, the tech stack is not just similar, but literally the same."
His hosted per-vCPU costs are about a third of EC2, and even after adding software to that figure, cloud data warehouses cost 2-4x more. It's not a small amount of money.
That's fair, I generally think about it as it relates to data stuff, which tends to be you can either 1) run some open source stuff on your own hardware that you have to maintain entirely on your own, or 2) push a button and it's done for you. So SaaS vs self-hosted. But if you're running real web infrastructure, you don't buy SaaS to do it; you just manage the same services you'd run yourself, but you manage them on different machines.
Can the Vision be as simple as creating the "organizational nervous system"? To create the next generation of AI-integrated companies where data is the electricity flowing through the body and computation pathways. There are Type I's out there. But where?
I think that's exactly the sort of thing that makes this hard, actually. Like, sure, an organization nervous system sounds great, but what IS it? How do I experience it? If I were someone who wanted to make that possible, what would I do to advance the cause?
I imagine people would come up with all sorts of different answers to those questions. So we end up pulling on different directions and pursuing different goals.
But if we had a way to understand what that meant - eg, the vision pro sizzle real, that shows the nervous system in our daily lives - we'd be more likely to all converge to the same place.
Appreciate the perspective Benn...to finding more of those Type 1's, who fall in love with the problem being solved, and less so with their current solution on the table. 🍻
The funny thing is, the type IIs should be *less* married to whatever they make, since it's just a means to an end. But it does seem like they actually get more attached. Maybe because they feel like they have to make it work, whereas type Is are more willing to make the hard choices that better serve the goal.
That's totally fair on the feeling of having to make it work. Think you hit the nail on the head with the hard part being the reliance on others to help "collectively" build the ecosystem, especially when all the oars aren't rowing in the same direction. For the consumer, it's like buying from the a la carte menu vs. the price fixe menu. There's times were it makes sense, but there's times where it totally doesn't from a value perspective. If everyone has to get their piece of the pie in the a la carte, it can price the consumer out of the market. That's where the price fixe menu may provide better value by bundling, even though it may not be as flexible in its choices.
Yeah, 100%. I think this is something that a lot of startups forget, that the buying process is part of the user experience too. And that's where Microsoft wins—they make it really really easy to buy their stuff. (I wrote something else about this back in the day: https://benn.substack.com/p/case-for-consolidation)
Thanks for sharing...another good read. Appreciate your directing the newb to your prior articles. 😀
I think that's where Tristan's recent article also got it right. As the incumbents have become more cloud-native, while also remaining feature-rich (one could even argue "bloated" in some cases), the dynamic of the competitive landscape for the collection of tools has changed. It can be a lot more expensive to tear up and start over than it is to stick with the incumbent and that one throat to choke as they evolve. That's where (to your point) that Type 1 focus on solving problems is essential. What's the 10x value driver that will compel folks to move?
For sure. (And I assume everyone here is new, because I doubt I'm tolerable for that long.)
And yeah, agreed. I literally just sent this email to a vendor who was pitching our team:
--
I asked people about this, and it doesn't seem like a need at the moment. They've got some tools for this already that work pretty well...So it sounds like people understand the value; it's just that there's a pretty big switching cost.
--
Buying something new, switching to it, etc - it's a huge cost. And even more so when you've already got a relationship with Microsoft, and you'll never replace everything that you buy from them.
I must be a glutton for punishment after Chill Data Summit. 😉
Seriously, you hit it there...it's really hard to get people to tool switch, especially if there's a bunch of embedded code in prod. The incumbent has to really mess up the relationship in somwe way. It's partially the reason we sold into the business with end-to-end solutions in my former role. They generally held the budget and didn't care as much about how the sausage was made. It's a bit more difficult of an existence though because it requires somewhat expensive industry vertical chops. You have to be selective on the target industries based on the serviceable obtainable market.
palantir's got a real well integrated vision for how all the pieces fit together in both an analytical and operational capacity
abhi sivasailam's described the most compelling one i've heard in an analytical capacity
clearly not designed to be a point solution but rather a series of pieces that come together to form a whole larger than the sum of its parts (as opposed to msft fabric)
I don't know anything about palantirs, though the original post I wrote about this idea in 2021 was partly inspired by the tools inside Uber and Airbnb, which I suspect are somewhat similar: https://benn.substack.com/p/the-modern-data-experience
On Ahbi's project, I know about soma, though I'd think of that as pretty different? It seems more like a framework for creating industry wide standards for metrics, and not a prescription for how those metrics make people's lives any different?
MDS tools have been solving the same technical problems on their own. Notably, integrations. Data movement, orchestration, transformation, quality, bi tools... all need integrations for each data storage system... And each tool solved it from scratch, with no common ground. Let me explain, when I worked for a BI tool, we wrote integrations to read data from data storages. When I worked for a data movement tool, we wrote connectors to read and write data from data storages. When I worked for a data quality tool, we wrote connectors to execute queries (tests) on data storages. Didn't work for a data orchestration tool, but operators basically make API calls to external systems, notably data storages. The value is in the data, or metadata if the system is not a data storage per se. And the common thing, is making api calls to these systems: read data, write data, query data (execute tests), trigger actions (orchestration). If there was a common framework to interact with external systems, then I think data tool companies could be more ambitious with a clear vision focused around data. We are missing standards, and a vision like you said. A standard to connect to any external system and be able to read, write, query and trigger actions. Then we need user experiences on top of that. And probably also a common standard on what to build user experiences.
Yeah, this has always bothered me too. Everyone builds connectors, or charts, or query interfaces, or whatever else. The whole stack is _sorta_ a bunch of components (a database, ETL, BI, etc), but each of those component is made from smaller, very duplicative components. Every product is like something on the taco bell menu - same ingredients, different arrangement.
I'd even venture in drawing a parallel with mobile apps: there's a lot of overlap between different apps, some even provide exactly the same value statement.
But it's all to the benefit of the marketplaces, not the users unfortunately.
The users have to navigate a sprawling landscape of ever-multiplicating products that all sell slightly differently a part of the same promise...
In our (data practitioners) case, the marketplaces are the (data) cloud providers which only stand to benefit from such a wide array of vendors all implementing something on top of their cloud.
I think there's a big chance we're going to witness exactly the same scenario play out with the GPTs marketplaces.
There seems to be a stronger open source core this time around, but will that be enough?
That's true, though I think there are two very big difference between data tools and apps: 1) Apps are cheap or free, and 2) they don't really need to work together. Data tools cost a lot to make, so they tend to be pretty expensive to buy. And, they all have to work together; you've got to actually integrate them into each other. So while a messy app ecosystem is is annoying, a messy data ecosystem is borderline nonfunctional.
Also, any outsider can look at Matt Turck's Machine learning AI, and data landscape and think - how in the, what?, am I to make sense of this? There's no connections illustrated between all of these products, and even if there was, it would look like every conspiracy theorist's pictures with string mood board. Hard to make sense of any gathering point in the chaos indeed.
My search turned up 2021's - how many products can one person know, have skills in, and even further, know now to fit it all together? https://mattturck.com/wp-content/uploads/2021/12/2021-MAD-Landscape-v3.pdf)
I audibly laughed when I got to " Anyway, speaking of the modern data stack." I think a fair amount of us that have been around the block a while can see exactly what you're highlighting so succinctly in this essay. Every conference I wind up going to, I keep thinking, aren't these the same sorts of problems we've been trying to "solve" for the last 20 some odd years I've been doing this, just re-packaged against the latest in the Gartner hype cycle? Having a Schelling point has to be something that is at the front of whatever evolution the modern data stack turns into. In a way, it's that "so what?" we're constantly grappling for in the majority of the solutions we're trying to deliver.
I was recently reading yet another article about how data teams aren't listened to, and how we can convince stakeholders to pay attention to us, and I think I've started to take an even more cynical turn on all of this, to be honest. I'm starting to think the entire approach we've taken to data (not just the tooling, but how we attempt to apply it to businesses) is pretty broken.
Like, If you make a product (in this case, the output of a data team) and nobody wants to buy it, for years, and you have to keep "training people how to use it," and they keep trying to work around you, and this happens, over and over and over again, until people who are selling the product get frustrated in quit...maybe it's the product you're selling?
Two thoughts - I don’t know him personally - but I think Tristan is one of the type 1 founders - and I agree there are not many in the MDS ecosystem. I really respect what dbt has done around mission and ecosystem.
I think a phase II of this would be getting engineers / developers AND data scientists full buy in for the MDS or analytics stack - or whatever we are calling it now. If those groups actually started fully adopting snowflake and dbt and other tools in the space I think there is a huge second wave of growth. (I know some did - but I think a lot did not). Then the promise of a unified data platform with data you trust to run your business on could actually be more true - and that was always my north star goal. Plus the added benefit of all your data in one place to hook up to AI stuff is a bonus - I think…
I think I half buy the second part of that. It seems possible, though I'm increasingly starting to think it's also possible that the entire arrangement - like, analytics, as a thing - doesn't really work. Not never, and that doesn't mean _data_ isn't useful, but I'm not sure that we'll ever get to a point where we have this whole data democratization utopia (or anything even close to it) that we've been talking up for a long time.
Ya - I get that too, I go back and forth. I also wonder if companies will be trying more dual role things - like let’s higher a sales manager and then buy an AI data tool - now we don’t need a sales analyst. I suspect this will happen and think it typically won’t work out well. AI seems like it will encourage armies of generalist - which might become its own problem…
Yeah, I'm not sure that will work either. I think I'm slowly going even further than that though, and wondering if, like, "insights" or whatever are actually there. My question is becoming, is analytics a failure? Not the way we do it, but the whole exercise?
Like do we really just need accounting, marketing,sales, product etc and not analytics?
It was this post that did me in. It's a perfectly good post; there's nothing about it specifically that bad or anything. But it's just...the same thing people have been trying to do forever? Like, we're on year 10 of begging our customers to buy our product, of trying to teach them how to use it, of telling them they don't understand how valuable it could be, and all of that. At some point, doesn't it just seem more likely that what we're selling isn't very useful??
https://petrjanda.substack.com/p/elevate-the-role-of-analytics-in
Ha, ya - you gotta be open to that possibility - for sure. I actually had a discussion this week with a friend who thinks it’s a generational problem and that he doesn’t think Gen X will make the leap to be more “data driven” - I’m not so convinced that’s the issue. I think part of the issue is selling such flexible un-opinionated tools.
Data Dog is a great analytics tool for example - but it clearly went after a market and build out specific purpose built features for security, troubleshooting errors, monitoring, etc. Maybe thats part of the solution.
IMHO be “bold and push” won’t get us there…
Stands up.
Starts slow clap.
Excellent critique. I've seen half a dozen different ways to construct a data architecture, each using a different set of vendors, a different approach to common problems, and a myriad of permutations of the MDS.
I found the use of the Schelling point as a "test" for the viability of a product / idea very compelling. It's telling that the MDS seems to be failing in that area. We must take a step back and reconsider.
I'm of the optimistic bent that data is a thing. That data at the enterprise scale, especially the global / multi-national enterprise scale, is a both a hard thing but a thing of immense value. Knowing the number of pumps in operation in an energy company is a good and necessary thing, and connecting that both to the maintenance records as well as to the revenue projections should be possible, maybe even easy. But for some reason our MDS in its current incarnation has struggled to solve the core challenges of the modern enterprise.
The journey must continue and this is a marker of a critical waypoint.
I appreciate that. And yeah, that's the nagging thing to me, is it sure *feels* like this should be useful? But something about it seems to keep coming up short.
My nagging concern is that these solutions are driven by technologists, based on their slightly too abstract point of view. I wonder what a data stack or data platform would look like that was driven only by a specific domain's business requirements?
My feeling is that the technologists are off just enough in defining the abstractions that it restricts their ability to understand/address the true problem. We technologists tend to be enamored by our patterns and frameworks, as if they are the thing. Too often we hijack the requirements discussion with our technical concerns, crowding out and obfuscating the actual business problems.
That raises a really interesting question to me - why haven't any non-data people started data companies?
Like, I sorta get it; it doesn't feel like your area of expertise, or whatever. But at the same time, it also feels like it's possible that they haven't because they just don't need what it's offering?
I think that they have, but we don't see them as data solutions. They are domain-specific applications with complex data requirements. These are the things we're often making the connectors for. But they tend to be focused on a specific set of problems in the domain, and as a result it is left to the company (or their consultants) to do the integration work with their other silo'd systems.
These types of applications would never pass muster with proper technologists, and tbh they are often pedestrian in their technical architectures and interfaces. But they work despite the lack of innovation (maybe that's a feature?) and can be cash cows.
That's probably true - we don't see them as data companies, so they fly under our radar. I'm not sure if that's a good or bad thing though. On one hand, it could mean there's all this opportunity, if only people combined the utility of those tools with the technical capabilities of tools build by data people. On the other hand, maybe that means the technical capabilities don't really add that much? I don't know.
Definitely saw this in healthcare and insurance...some of the solutions ignored all the hard stuff. Was basically, if you can figure out how to harmonize, cleanse, and master your data into my model, i can spit out some whiz-bang visualizations for you.
The costs are the part that never got better. I've been "On the Cloud" since 2009, and I've been six months away from the true cost savings for the entire time. Until the last few years people were fairly rational about it, but lately the dynamic has become accelerationist, with execs spending millions on "cloud native" snake oil and blundering into layoffs.
DHH's recent Cloud Exit of 37signals' app hosting has been impressive, and I think the cost differentials are even bigger for the data space.
I pretty mixed on this. Though I think it's fair to say that SaaS products have a bunch of hidden and obfuscated costs (which was what this piece was basically about https://benn.substack.com/p/data-and-the-almighty-dollar), that also seems true for self-hosting? Like, to me, the biggest benefit of SaaS isn't necessarily the dollar cost, but the ease of management, and not having to deal with the headaches of self-hosting. Like, I'd probably be willing to pay *more* for SaaS?
That said, there's a danger with the ease of use, which is that it's easy to buy without really thinking that carefully about it. So maybe that actually eats away a lot of the gains.
My experience agrees with his statement that "80-90% of what you need to know to run your own hardware these days is the same as what you need to know to operate in the rental clouds. From containers to load balancers to monitoring to performance analysis, and a million other topics, the tech stack is not just similar, but literally the same."
His hosted per-vCPU costs are about a third of EC2, and even after adding software to that figure, cloud data warehouses cost 2-4x more. It's not a small amount of money.
That's fair, I generally think about it as it relates to data stuff, which tends to be you can either 1) run some open source stuff on your own hardware that you have to maintain entirely on your own, or 2) push a button and it's done for you. So SaaS vs self-hosted. But if you're running real web infrastructure, you don't buy SaaS to do it; you just manage the same services you'd run yourself, but you manage them on different machines.
Can the Vision be as simple as creating the "organizational nervous system"? To create the next generation of AI-integrated companies where data is the electricity flowing through the body and computation pathways. There are Type I's out there. But where?
I think that's exactly the sort of thing that makes this hard, actually. Like, sure, an organization nervous system sounds great, but what IS it? How do I experience it? If I were someone who wanted to make that possible, what would I do to advance the cause?
I imagine people would come up with all sorts of different answers to those questions. So we end up pulling on different directions and pursuing different goals.
But if we had a way to understand what that meant - eg, the vision pro sizzle real, that shows the nervous system in our daily lives - we'd be more likely to all converge to the same place.
Appreciate the perspective Benn...to finding more of those Type 1's, who fall in love with the problem being solved, and less so with their current solution on the table. 🍻
The funny thing is, the type IIs should be *less* married to whatever they make, since it's just a means to an end. But it does seem like they actually get more attached. Maybe because they feel like they have to make it work, whereas type Is are more willing to make the hard choices that better serve the goal.
That's totally fair on the feeling of having to make it work. Think you hit the nail on the head with the hard part being the reliance on others to help "collectively" build the ecosystem, especially when all the oars aren't rowing in the same direction. For the consumer, it's like buying from the a la carte menu vs. the price fixe menu. There's times were it makes sense, but there's times where it totally doesn't from a value perspective. If everyone has to get their piece of the pie in the a la carte, it can price the consumer out of the market. That's where the price fixe menu may provide better value by bundling, even though it may not be as flexible in its choices.
Yeah, 100%. I think this is something that a lot of startups forget, that the buying process is part of the user experience too. And that's where Microsoft wins—they make it really really easy to buy their stuff. (I wrote something else about this back in the day: https://benn.substack.com/p/case-for-consolidation)
Thanks for sharing...another good read. Appreciate your directing the newb to your prior articles. 😀
I think that's where Tristan's recent article also got it right. As the incumbents have become more cloud-native, while also remaining feature-rich (one could even argue "bloated" in some cases), the dynamic of the competitive landscape for the collection of tools has changed. It can be a lot more expensive to tear up and start over than it is to stick with the incumbent and that one throat to choke as they evolve. That's where (to your point) that Type 1 focus on solving problems is essential. What's the 10x value driver that will compel folks to move?
For sure. (And I assume everyone here is new, because I doubt I'm tolerable for that long.)
And yeah, agreed. I literally just sent this email to a vendor who was pitching our team:
--
I asked people about this, and it doesn't seem like a need at the moment. They've got some tools for this already that work pretty well...So it sounds like people understand the value; it's just that there's a pretty big switching cost.
--
Buying something new, switching to it, etc - it's a huge cost. And even more so when you've already got a relationship with Microsoft, and you'll never replace everything that you buy from them.
I must be a glutton for punishment after Chill Data Summit. 😉
Seriously, you hit it there...it's really hard to get people to tool switch, especially if there's a bunch of embedded code in prod. The incumbent has to really mess up the relationship in somwe way. It's partially the reason we sold into the business with end-to-end solutions in my former role. They generally held the budget and didn't care as much about how the sausage was made. It's a bit more difficult of an existence though because it requires somewhat expensive industry vertical chops. You have to be selective on the target industries based on the serviceable obtainable market.
Fun chat...thanks for indulging me in the convo.
palantir's got a real well integrated vision for how all the pieces fit together in both an analytical and operational capacity
abhi sivasailam's described the most compelling one i've heard in an analytical capacity
clearly not designed to be a point solution but rather a series of pieces that come together to form a whole larger than the sum of its parts (as opposed to msft fabric)
I don't know anything about palantirs, though the original post I wrote about this idea in 2021 was partly inspired by the tools inside Uber and Airbnb, which I suspect are somewhat similar: https://benn.substack.com/p/the-modern-data-experience
On Ahbi's project, I know about soma, though I'd think of that as pretty different? It seems more like a framework for creating industry wide standards for metrics, and not a prescription for how those metrics make people's lives any different?