Really great analysis. I've been thinking much the same for a while now and also chewing on how, as we get better at understanding what it takes to create that utopian, unified data platform, and the lengthy list of tools and capability it's gonna need. How do we encourage better behaviours and architectures when everything is so damn expensive? We have an industry with a rich history of gouging its customers and making every little thing ultra expensive. I can only hope that we see what happened in the software world over a decade ago, and see a surge of free open source tooling driven by engineers fed up with doing everything so badly. Given the influx of software engineers into the data field, I am crossing my fingers.. As Dave points out in his comment, there is value in decoupling a semantic layer for data from visualisations and dashboards... It also goes far beyond just that.
Agreed, though I've heard some people talk recently about how the software analogy might not be a good one. I think a lot of people (me included) fall back to it because it's an easy comparison and it definitely fits in some ways. But it might be a bad fin in other ways, and open source might be one of those ways. So much data tooling now is oriented around applications - and so many people want to pay for someone to take care of the headache of running it - it's not clear to the open source elements of the industry will ever build enough momentum to take over.
I get your point. The barrier to full open source products in data is real and I agree with the self management part - especially in enterprise environments. I suppose I live in hope as well as being content when there is a managed option based on open source whether that's an established product or a company open-sourcing its new tools. Maybe I just want to believe :)
I think dbt will be an interesting bellwether. It's 1) popular, 2) open-source, and 3) pretty easy to self-host (as opposed to something like spark). If that ends up being more SaaS product than open-source tool, then it doesn't feel like other tools have much hope.
Hey Jared, thanks for the invite! I'm interested - can you reach out over email, linkedin, or Twitter (all linked here: https://benn.substack.com/about)? Thanks!
Hi Benn. I do believe that this announcement is an important one because it explicitly acknowledges that the semantic layer has value separate and distinct from the consumption layer. I started AtScale in 2013 with this core concept in mind and it's great to finally see the large ecosystem players come to the same realization. Have no doubt, though, that Google's intentions are to drive customers to BigQuery and consequently, GCP. Their investment in BI Engine is another example of trying to compete with Snowflake by positioning them as just a dumb data store.
Yeah, I'm sure that's the long term goal. This seems like a bit of indirect way to get people there, though the BI Engine could be an interesting path, if the LookML metrics layer starts to do a bunch of fancy cacheing stuff through the BI Engine.
Interesting analysis. Some pieces do however feel a bit off to me because Looker is marketed and sold as a premium, powerful and yet consumer friendly BI platform first. It is a very expensive product and they had the means to put such a heavy price tag because they closed the gap between a powerful data modelling layer and best in class visualisations and reporting tools. Shifting their strategy to focus on data modelling would put them right next to open source solutions that solve those problems very elegantly and essentially for free or as a dedicated online service (e.g. DBT Core & DBT Cloud).
As you suggest, it does seem to me as a way to "align with the horizontal grooves the industry is carving" in order to stay close and a relevant alternative when compared to the competition. But I don't see that as a shift of strategy.
I think that all depends on how they price. That's the big unknown for me on this. If you have to pay for "full" Looker to use this integration, I agree - it's not so much a shift in strategy than a way to help out folks who already use both Looker and Tableau. But, if they begin to offer ways to buy LookML on its own (presumably at a discounted price), it gets a lot more interesting.
Really great analysis. I've been thinking much the same for a while now and also chewing on how, as we get better at understanding what it takes to create that utopian, unified data platform, and the lengthy list of tools and capability it's gonna need. How do we encourage better behaviours and architectures when everything is so damn expensive? We have an industry with a rich history of gouging its customers and making every little thing ultra expensive. I can only hope that we see what happened in the software world over a decade ago, and see a surge of free open source tooling driven by engineers fed up with doing everything so badly. Given the influx of software engineers into the data field, I am crossing my fingers.. As Dave points out in his comment, there is value in decoupling a semantic layer for data from visualisations and dashboards... It also goes far beyond just that.
Agreed, though I've heard some people talk recently about how the software analogy might not be a good one. I think a lot of people (me included) fall back to it because it's an easy comparison and it definitely fits in some ways. But it might be a bad fin in other ways, and open source might be one of those ways. So much data tooling now is oriented around applications - and so many people want to pay for someone to take care of the headache of running it - it's not clear to the open source elements of the industry will ever build enough momentum to take over.
I get your point. The barrier to full open source products in data is real and I agree with the self management part - especially in enterprise environments. I suppose I live in hope as well as being content when there is a managed option based on open source whether that's an established product or a company open-sourcing its new tools. Maybe I just want to believe :)
I think dbt will be an interesting bellwether. It's 1) popular, 2) open-source, and 3) pretty easy to self-host (as opposed to something like spark). If that ends up being more SaaS product than open-source tool, then it doesn't feel like other tools have much hope.
Hi Ben, I’d like to have you join us on Data Sharks. Let me know if you’re interested. https://youtube.com/playlist?list=PLIBwmCV8YuLPOKsqwovvBgMhBDwYYyB85
Hey Jared, thanks for the invite! I'm interested - can you reach out over email, linkedin, or Twitter (all linked here: https://benn.substack.com/about)? Thanks!
Hi Ben, I just sent you a LinkedIn invite. We can exchange emails there.
Hi Benn. I do believe that this announcement is an important one because it explicitly acknowledges that the semantic layer has value separate and distinct from the consumption layer. I started AtScale in 2013 with this core concept in mind and it's great to finally see the large ecosystem players come to the same realization. Have no doubt, though, that Google's intentions are to drive customers to BigQuery and consequently, GCP. Their investment in BI Engine is another example of trying to compete with Snowflake by positioning them as just a dumb data store.
Yeah, I'm sure that's the long term goal. This seems like a bit of indirect way to get people there, though the BI Engine could be an interesting path, if the LookML metrics layer starts to do a bunch of fancy cacheing stuff through the BI Engine.
Interesting analysis. Some pieces do however feel a bit off to me because Looker is marketed and sold as a premium, powerful and yet consumer friendly BI platform first. It is a very expensive product and they had the means to put such a heavy price tag because they closed the gap between a powerful data modelling layer and best in class visualisations and reporting tools. Shifting their strategy to focus on data modelling would put them right next to open source solutions that solve those problems very elegantly and essentially for free or as a dedicated online service (e.g. DBT Core & DBT Cloud).
As you suggest, it does seem to me as a way to "align with the horizontal grooves the industry is carving" in order to stay close and a relevant alternative when compared to the competition. But I don't see that as a shift of strategy.
I think that all depends on how they price. That's the big unknown for me on this. If you have to pay for "full" Looker to use this integration, I agree - it's not so much a shift in strategy than a way to help out folks who already use both Looker and Tableau. But, if they begin to offer ways to buy LookML on its own (presumably at a discounted price), it gets a lot more interesting.