Oh nice. There was a bar in DC called Big Board that also attempted this, but it was kinda dumb and gimmicky. They prices didn't move very much, and they seemed to mostly do it for the "crash" thing, so that they could have a 15 minute sale or something. I want the prices to go up like 300%
Aug 25, 2023·edited Aug 25, 2023Liked by Benn Stancil
I think you said it - the thing that makes this new pricing model work is the dbt Cloud user experience / feature set. Can it be compelling enough? Can they nail developer experience where not using it feels much less comfortable. I personally would LOVE to see DBT acquire Dagster. If they don't, Dagster has a very good product from what I have seen and Dagster + DBT Core is pretty compelling to me.
Yeah, and that feels like the real risk to dbt. It's not from people using dbt core and their own homemade scheduler; it's from some other company selling something on top of dbt core, and using it's traction to vault past dbt labs.
Aug 26, 2023·edited Aug 26, 2023Liked by Benn Stancil
Indeed. Dagster still feels pretty complicated to me even if your using their cloud product (which I have only spent a very small amount of time with so far) - so dbt has a chance to build something really good in the orchestration world that balances sophistication and ease of use.
There’s also a kind of paradox there, where it seems possible to me that if you’re trying to sell something that’s based on OSS, you actually want the open source thing it to be complicate. People don’t want to deal with Spark because it’s a huge pain, so Databricks can built a huge business around it. dbt’s biggest challenge might be that it’s too easy to use, so people don’t need the hosted thing.
Huh - ya very true. Incentives are aligned to create layers of complexity in the open source version and solve the complexity introduced in the cloud version. Lol.
Everything boils down to this. People want to get done what they need to, meeting certain criteria, with a reasonable balance between cost and effort.
It's not like pricing is an easy problem to solve, or that pricing is only set by malicious players. This is a hard problem to solve if you're not starting from scratch, as you pointed out. Lot of "smart data people" like to throw rocks and say they could have done this all better. And maybe they could. But as the adage goes, the key to failure is trying to please everyone.
That's my biggest question in all of this: What choice should they make here? It's easy to criticize it, because it's easy to be a critic, especially for pricing problems that always have some fundamental sounding flaw. But dbt's gotta do something - so what should they do?
I asked the same thing as I couldn't imagine what else they could have gone with. I agree with you, there was nothing easy. I think the consumable thing is sensible, the issue is the lack of stickiness of the product. Made worse by
1. grandfathering in their existing customers, so the change is slow to make an uptick in revenue
2. new market are spoiled for choice when it comes to dbt IDEs and CICD. What % of their possible TAM would they be on by now that the rest of the addressable market must be relatively small, and relatively educated (crossing the lifecycle chasm etc), now competing with tiny teams.
Point 2 seems like the big question to me. To what extent is the market actually tapped out or aware of all the choice? The startup world definitely is, I think, but are enterprise buyers? I really don't know, but I'd doubt it?
Agree, and enterprise buyers is tricky as they want the opposite of the MDS. I just read this interview below (disagree with some of it), and am trying to think what the appropriate tweet is, but I definitely agree with the premise that enterprise need (need, not want) integrated systems and the MDS and dbt are not that. This makes me think dbt is possibly tapped out in that direction too, or will find increasingly less willing buyer. https://signalfire.com/modern-data-stack-investor/
My thought is, maybe tangential, but 2 below addresses the question:
1. MDS is possible to achieve more than the sum of its parts, less constrained, more expressive, more suitable to smart teams.
2. Enterprise as a collective unit is less capable of being smart, and so needs its products to be more guard-railed.
After working as a post sales engineer for an enterprise data integration (like dbt+Snowflake+Fivetran in one), and the incredibly stupid things that I saw people do with it, and the stupid things I tried to do with it before I worked there was astounding. The primary reason was the person buying it was 3 rungs away from the team implementing it, and on the ground have entirely different priorities and enough distance to head in a different direction. This makes me think 2 is true, and makes the MDS largely incompatible with sprawlingly large enterprise teams.
It is not just the capability that enterprise is after, it is tail risk protection. Opposite is true for small teams running MDS, they are after the upside opportunity.
I used to have this bit that if you could use one signal to find prospective buyers for MDS tools, it’d be a measure of how ambitious a data team was. If it’s a team that wants to go home without being bothered, they’re a terrible buyer; if it’s a team that wants to do things and get recognized and all that, they’re a great buyer. I doubt that matches exactly (or even terribly well) to the enterprise vs startup, but it’s probably correlated.
(I do think the all-in-one thing is important for bigger companies though, which has kind of been a background rant on this blog for a while.)
I think that maps reasonably well, maybe a given for startup teams to be ambitious, but probably something there. I definitely agree with the IT orientation of the terrible buyer.
Microsoft Fabric could be to MDS what Microsoft Teams was to Slack. (you probably said that in that blog, but it would I guess be the likely tell that instead of dbt the uptake is all with something that is actually just Microsoft SSIS)
Fundamentally, the issue at hand is not one of structure, but one of magnitude. Customers weren't objecting to the pricing structure so much as the implied increase in price. Your quote about what is "required to justify their 2022 fundraising round" is the punchline and really all that needs to be said.
I think that's true, but only if you assume that dbt Cloud and Core are truly independent. But they're aren't - it's just the pricing structure (of which OSS is part) that makes them feel that way.
Here's another way to think about it. Suppose that, in addition to the pricing changes, dbt had done the super aggressive thing here, made dbt core closed source, and started hunting down people who used it without cloud (I don't think this is exactly possible or legal, but let's go with it). People would obviously be very upset, but I don't think people would actually say that dbt in its entirety isn't worth what it would cost them. The problem would be the bait and switch, and the feeling of being duped, or whatever. The value is there; dbt created it for people; the problem is how does dbt capture it.
To use the bar example, suppose that you build a bar on edge of a beautiful beach. The value is there to charge a huge amount for your drinks. But can you? That depends on if people see themselves as buying drinks from you, or drinks and a view.
We're obviously insane. SnowBuilder for Snowflake (patent pending) is unlimited usage for monthly subscription. Intent is Snowflake Marketplace. No way to do per seat pricing and usage pricing is also problematic.
The delivery costs us nothing and the direct usage costs to customer is negligible. Require XSM warehouse and subsecond execution.
What you might do with the results is another ,issue. Giving all your end users extensive libraries of named, plug and play query capabilities might increase your compute spend, hopefully with corresponding productivity and ROI😎
Stop giving people ideas! https://www.lbc.co.uk/news/stonegate-pub-chain-charge-punters-more-peak-times/
Ok, but the real story here is, there's a bar called "Slug and lettuce"???!
There's a typo "dbt Labs started changing by the drink" -> charging? Thanks for the content, amazing as always!
Otherwise you might be making some claims about the dbt Labs team I wasn't aware of . In that case do send them that Huberman link.
It’s those eight Muni Martinis, with each one the pricing model gets crazier and crazier.
Have you seen https://dowjonesbar.com/ in Barcelona? They don’t charge rent for the tables though
Oh nice. There was a bar in DC called Big Board that also attempted this, but it was kinda dumb and gimmicky. They prices didn't move very much, and they seemed to mostly do it for the "crash" thing, so that they could have a 15 minute sale or something. I want the prices to go up like 300%
I think you said it - the thing that makes this new pricing model work is the dbt Cloud user experience / feature set. Can it be compelling enough? Can they nail developer experience where not using it feels much less comfortable. I personally would LOVE to see DBT acquire Dagster. If they don't, Dagster has a very good product from what I have seen and Dagster + DBT Core is pretty compelling to me.
Yeah, and that feels like the real risk to dbt. It's not from people using dbt core and their own homemade scheduler; it's from some other company selling something on top of dbt core, and using it's traction to vault past dbt labs.
Indeed. Dagster still feels pretty complicated to me even if your using their cloud product (which I have only spent a very small amount of time with so far) - so dbt has a chance to build something really good in the orchestration world that balances sophistication and ease of use.
There’s also a kind of paradox there, where it seems possible to me that if you’re trying to sell something that’s based on OSS, you actually want the open source thing it to be complicate. People don’t want to deal with Spark because it’s a huge pain, so Databricks can built a huge business around it. dbt’s biggest challenge might be that it’s too easy to use, so people don’t need the hosted thing.
Huh - ya very true. Incentives are aligned to create layers of complexity in the open source version and solve the complexity introduced in the cloud version. Lol.
If you want to make money as a fireman, be an arsonist, I guess.
😂
Everything boils down to this. People want to get done what they need to, meeting certain criteria, with a reasonable balance between cost and effort.
It's not like pricing is an easy problem to solve, or that pricing is only set by malicious players. This is a hard problem to solve if you're not starting from scratch, as you pointed out. Lot of "smart data people" like to throw rocks and say they could have done this all better. And maybe they could. But as the adage goes, the key to failure is trying to please everyone.
That's my biggest question in all of this: What choice should they make here? It's easy to criticize it, because it's easy to be a critic, especially for pricing problems that always have some fundamental sounding flaw. But dbt's gotta do something - so what should they do?
I asked the same thing as I couldn't imagine what else they could have gone with. I agree with you, there was nothing easy. I think the consumable thing is sensible, the issue is the lack of stickiness of the product. Made worse by
1. grandfathering in their existing customers, so the change is slow to make an uptick in revenue
2. new market are spoiled for choice when it comes to dbt IDEs and CICD. What % of their possible TAM would they be on by now that the rest of the addressable market must be relatively small, and relatively educated (crossing the lifecycle chasm etc), now competing with tiny teams.
Point 2 seems like the big question to me. To what extent is the market actually tapped out or aware of all the choice? The startup world definitely is, I think, but are enterprise buyers? I really don't know, but I'd doubt it?
Agree, and enterprise buyers is tricky as they want the opposite of the MDS. I just read this interview below (disagree with some of it), and am trying to think what the appropriate tweet is, but I definitely agree with the premise that enterprise need (need, not want) integrated systems and the MDS and dbt are not that. This makes me think dbt is possibly tapped out in that direction too, or will find increasingly less willing buyer. https://signalfire.com/modern-data-stack-investor/
My thought is, maybe tangential, but 2 below addresses the question:
1. MDS is possible to achieve more than the sum of its parts, less constrained, more expressive, more suitable to smart teams.
2. Enterprise as a collective unit is less capable of being smart, and so needs its products to be more guard-railed.
After working as a post sales engineer for an enterprise data integration (like dbt+Snowflake+Fivetran in one), and the incredibly stupid things that I saw people do with it, and the stupid things I tried to do with it before I worked there was astounding. The primary reason was the person buying it was 3 rungs away from the team implementing it, and on the ground have entirely different priorities and enough distance to head in a different direction. This makes me think 2 is true, and makes the MDS largely incompatible with sprawlingly large enterprise teams.
It is not just the capability that enterprise is after, it is tail risk protection. Opposite is true for small teams running MDS, they are after the upside opportunity.
I used to have this bit that if you could use one signal to find prospective buyers for MDS tools, it’d be a measure of how ambitious a data team was. If it’s a team that wants to go home without being bothered, they’re a terrible buyer; if it’s a team that wants to do things and get recognized and all that, they’re a great buyer. I doubt that matches exactly (or even terribly well) to the enterprise vs startup, but it’s probably correlated.
(I do think the all-in-one thing is important for bigger companies though, which has kind of been a background rant on this blog for a while.)
I think that maps reasonably well, maybe a given for startup teams to be ambitious, but probably something there. I definitely agree with the IT orientation of the terrible buyer.
Microsoft Fabric could be to MDS what Microsoft Teams was to Slack. (you probably said that in that blog, but it would I guess be the likely tell that instead of dbt the uptake is all with something that is actually just Microsoft SSIS)
Fundamentally, the issue at hand is not one of structure, but one of magnitude. Customers weren't objecting to the pricing structure so much as the implied increase in price. Your quote about what is "required to justify their 2022 fundraising round" is the punchline and really all that needs to be said.
I think that's true, but only if you assume that dbt Cloud and Core are truly independent. But they're aren't - it's just the pricing structure (of which OSS is part) that makes them feel that way.
Here's another way to think about it. Suppose that, in addition to the pricing changes, dbt had done the super aggressive thing here, made dbt core closed source, and started hunting down people who used it without cloud (I don't think this is exactly possible or legal, but let's go with it). People would obviously be very upset, but I don't think people would actually say that dbt in its entirety isn't worth what it would cost them. The problem would be the bait and switch, and the feeling of being duped, or whatever. The value is there; dbt created it for people; the problem is how does dbt capture it.
To use the bar example, suppose that you build a bar on edge of a beautiful beach. The value is there to charge a huge amount for your drinks. But can you? That depends on if people see themselves as buying drinks from you, or drinks and a view.
We're obviously insane. SnowBuilder for Snowflake (patent pending) is unlimited usage for monthly subscription. Intent is Snowflake Marketplace. No way to do per seat pricing and usage pricing is also problematic.
The delivery costs us nothing and the direct usage costs to customer is negligible. Require XSM warehouse and subsecond execution.
What you might do with the results is another ,issue. Giving all your end users extensive libraries of named, plug and play query capabilities might increase your compute spend, hopefully with corresponding productivity and ROI😎