Lots of good thoughts here (doesn't DB buying dbt turn that into scary vendor lock-in hell?) - but what most impresses me is the nod to the Moth Joke. Go ahead, murder someone, you're in my good book for all time Ben Stancil.
On the lock-in thing, it probably would to some extent, and I suspect that's one of the big reasons dbt wouldn't go for it. But as a business proposition for Databricks, that's part of the appeal. (Plus, I think there's a middle ground in there, where dbt can still work anything, but has more bells and whistles with Databricks.)
It's my fav joke of all time. Probably 90% of its greatness is how he delivered it.
[I think lock-in is overhyped and unavoidable and I was mostly being sarcastic about it. People hype about its supposed evils, but it's just another risk to manage.]
if a DB bought dbt, there would likely be a forked Apache-run spinoff (so you could still use it as an open source product and avoid the vendor lock-in)
I'm not even sure that's necessary. dbt could continue to support the open-source version. Databricks would just want to build more integrations with it, make the dbt-databricks driver really strong, and add various things cloud services to it like better scheduling that would be part of the databricks package. But I don't see why dbt core couldn't stay as it is.
That Databricks acquisition of dbt is interesting and echoes what we see across our teams. The BI team uses Snowflake with dbt while the data engineering team uses Databricks. To your point this is a way of jumping over that "Databricks is for data science" perception.
For whatever reason, they really haven't been able to get away from that brand (though I guess in fairness, I'm not entirely sure they've really been trying to?)
Your logic behind DB buying dbt reminds me of Snowflake's Streamlit acquisition: getting instant access to a large Python community which is expected to run AI workloads on Snowflake's compute engine.
Hey Benn, excellent summary. I'm wondering if it's alright to draw comparisons with the Redash acquisition. I've always believed that the product is undervalued and has the potential to go beyond just BI. I still use the open-source versions today. However, with the acquisition, it simply disappeared into Databricks' ecosystem and was unable to stand out or have a say in the roadmap.
I don't any of the internal details, but I'd guess that'd be a pretty different thing (or at least, would start as a different thing). Redash was a lot smaller business than dbt, so it's not that surprising that it'd get swallowed up by Databricks. I doubt Databricks ever saw that as a full-on selling point either, but more as an easy way to get started.
Obviously, the same thing could happen to dbt, but if nothing else, I'd think that Databricks would try a lot harder to keep it from happening. Given the price, they'd be buying it for its marketing position, so I'd think they'd have a pretty strong incentive to protect that.
Certainly, the scale may be dissimilar, but in my opinion, Redash has vended out prematurely. It would be an enhanced ingress spot for the semantic search product if it was less reliant on the Databrick ecosystem.
It goes without saying that this does not take into account the motivation of the Redash founders - as they self-financed the venture and eventually exited with a handsome profit margin.
Amusing and thought-provoking as always. I confess, I also would love to see DBT labs and DataBricks hook up. Though I wonder if that’s more emotional than rational...
Lots of good thoughts here (doesn't DB buying dbt turn that into scary vendor lock-in hell?) - but what most impresses me is the nod to the Moth Joke. Go ahead, murder someone, you're in my good book for all time Ben Stancil.
On the lock-in thing, it probably would to some extent, and I suspect that's one of the big reasons dbt wouldn't go for it. But as a business proposition for Databricks, that's part of the appeal. (Plus, I think there's a middle ground in there, where dbt can still work anything, but has more bells and whistles with Databricks.)
The moth joke though. So good.
It's my fav joke of all time. Probably 90% of its greatness is how he delivered it.
[I think lock-in is overhyped and unavoidable and I was mostly being sarcastic about it. People hype about its supposed evils, but it's just another risk to manage.]
100%. Even just holding on for that long, it's like a staring contest, how do you not crack?
[ And agreed, actually. I think it's often more of a marketing pitch than something customers really care about. ]
Vendor: You should buy my product to avoid vendor lock in.
Prospect: If I build my application with your product, isn't that vendor lock in?
Vendor 1: Yes, but we're nicer.
if a DB bought dbt, there would likely be a forked Apache-run spinoff (so you could still use it as an open source product and avoid the vendor lock-in)
I'm not even sure that's necessary. dbt could continue to support the open-source version. Databricks would just want to build more integrations with it, make the dbt-databricks driver really strong, and add various things cloud services to it like better scheduling that would be part of the databricks package. But I don't see why dbt core couldn't stay as it is.
3 cheers for Fish!
ok let's not get too carried away
That Databricks acquisition of dbt is interesting and echoes what we see across our teams. The BI team uses Snowflake with dbt while the data engineering team uses Databricks. To your point this is a way of jumping over that "Databricks is for data science" perception.
For whatever reason, they really haven't been able to get away from that brand (though I guess in fairness, I'm not entirely sure they've really been trying to?)
Your logic behind DB buying dbt reminds me of Snowflake's Streamlit acquisition: getting instant access to a large Python community which is expected to run AI workloads on Snowflake's compute engine.
Yeah, though I'd argue it'd go even further, since dbt is both a big community *and* a really popular product.
Hey Benn, excellent summary. I'm wondering if it's alright to draw comparisons with the Redash acquisition. I've always believed that the product is undervalued and has the potential to go beyond just BI. I still use the open-source versions today. However, with the acquisition, it simply disappeared into Databricks' ecosystem and was unable to stand out or have a say in the roadmap.
I don't any of the internal details, but I'd guess that'd be a pretty different thing (or at least, would start as a different thing). Redash was a lot smaller business than dbt, so it's not that surprising that it'd get swallowed up by Databricks. I doubt Databricks ever saw that as a full-on selling point either, but more as an easy way to get started.
Obviously, the same thing could happen to dbt, but if nothing else, I'd think that Databricks would try a lot harder to keep it from happening. Given the price, they'd be buying it for its marketing position, so I'd think they'd have a pretty strong incentive to protect that.
Certainly, the scale may be dissimilar, but in my opinion, Redash has vended out prematurely. It would be an enhanced ingress spot for the semantic search product if it was less reliant on the Databrick ecosystem.
It goes without saying that this does not take into account the motivation of the Redash founders - as they self-financed the venture and eventually exited with a handsome profit margin.
I don't think I understand what you mean by they "vended out prematurely." They sold the company too soon?
And I'm not sure why it's be a particularly good product for semantic search, at least relative to a number of other similar tools out there.
Amusing and thought-provoking as always. I confess, I also would love to see DBT labs and DataBricks hook up. Though I wonder if that’s more emotional than rational...