So let me be more specific. The way I’m seeing Data Factory being used (a component of Fabric) is taking SSIS packages and now running them in the cloud. That’s not the only way to use data factory - but as long as you don’t have to rewrite SSIS jobs I imagine many people won’t... but then you have the limitations of the tech that was already in place - with some benefit of now having a cloud runner. PowerBI is also part of fabric - and the reason I hear from others as to why they use it is ALWAYS “because it was cheaper” not better. Plus the cheaper was just in LICENSING cost (not counting other costs).
The bizarre part - Microsoft Excel is a tool people still truly love. I’m sure MS product teams would love to bottle that Excel lovin’ and spread to other MS data products - but IMO that hasn’t happened.
Ah, gotcha. So it's all kind of a convenience and price thing, with maybe a bit of gloss to polish it up some. But, not some huge revolution of new ideas. Which, to be fair, if you're MSFT, it's not at all clear that you need that. Just getting people to buy the things you've already built probably isn't a bad strategy.
Yes "Analyze in Excel" is the killer feature in the Fabric/PowerBI stack. Once this becomes more version controlled and extensible on the backend (on roadmap for PowerBI team I believe!) I could totally see this stack as the primary impediment to DBT clouds semantic layer + all the other BI tools supporting it gaining any meaningful market share among established orgs. I'm saying this as a huge fan of how the dbt-metricflow Google sheets integration works too and how fast they got it up and running. There are just too many excel users in the world.
I read this and worry that my concerns are real that the data world has been hijacked by people who would rather create governance frameworks and controls, than find (or allow others to find) insights and understanding in data. I think this is in large due to its promotion to ‘the new oil’ and the explosion in roles which are often siloed in the data stack. The increase in status leads to greater scrutiny, and the increase in silos increases the likelihood of errors and mistakes due to lack of insight…which leads again to greater scrutiny.
In governance specifically, I think there is a data lifecycle which starts being produced by business process, when that process is entered into an application, and via ingestions and transformations into reporting, through influencing and into action/decisions. So far regulatory governance approaches I’ve seen only inspect the problem of (a small amount of the ingestion,) transformations and models, but still manages to miss the two key ends of the process. I believe this is one reason why it’s boring - it seems like overhead that is added doesn’t solve the key issue of knowledge gaps in understanding what processes, developer choices, and manager idiosyncrasies help data be productive and valuable
That's an interesting point, and seems probably true? Though not necessarily for nefarious reasons or some sort of intentional capture, but because those things are easier to sell?
Back in the day, we at Mode competed a bunch with Looker. It wasn't an easy sell, because our story was "we help people analyze data and make better decisions," and Looker's story was "we help you set up a governance model so that your dashboards aren't consistent." Though I still believe that the value of a data team is largely in the former story, it's both squishy and hard to achieve. The latter story is both a lot more concrete, and feels like something that can be *completed.* You can build the models; you can check that box. It's a solvable technical problem, rather than a ongoing human problem. And I think there's something much more appealing about that, to both buyers and to people building software.
Ya that’s my impression so far. I’m trying to be open minded... but ya. Oh and absolutely if MS built something extraordinary that they just couldn’t sell to their enterprise customers because it was too big of a change or too different - that certainly doesn’t work. Plus to your point I don’t think they are asking for something completely new and different.
My Microsoft Fabric comments - I have not spent much time - however what I have seen are some existing tools being rebranded as MS Fabric. I took a 5 year break from the Microsoft stack... As I dive back in things feel waay too similar to the SSXSs of old. It feels like the same philosophy with some shiny sprinkled on top. I know there are *some* net new tools. but I struggle to think MS is going to all of a sudden become a leader here....
Have you used it much? I've heard from people who are optimistic about in theory, but heard very little from people who've actually spent much time doing real work in it.
So let me be more specific. The way I’m seeing Data Factory being used (a component of Fabric) is taking SSIS packages and now running them in the cloud. That’s not the only way to use data factory - but as long as you don’t have to rewrite SSIS jobs I imagine many people won’t... but then you have the limitations of the tech that was already in place - with some benefit of now having a cloud runner. PowerBI is also part of fabric - and the reason I hear from others as to why they use it is ALWAYS “because it was cheaper” not better. Plus the cheaper was just in LICENSING cost (not counting other costs).
The bizarre part - Microsoft Excel is a tool people still truly love. I’m sure MS product teams would love to bottle that Excel lovin’ and spread to other MS data products - but IMO that hasn’t happened.
Ah, gotcha. So it's all kind of a convenience and price thing, with maybe a bit of gloss to polish it up some. But, not some huge revolution of new ideas. Which, to be fair, if you're MSFT, it's not at all clear that you need that. Just getting people to buy the things you've already built probably isn't a bad strategy.
Yes "Analyze in Excel" is the killer feature in the Fabric/PowerBI stack. Once this becomes more version controlled and extensible on the backend (on roadmap for PowerBI team I believe!) I could totally see this stack as the primary impediment to DBT clouds semantic layer + all the other BI tools supporting it gaining any meaningful market share among established orgs. I'm saying this as a huge fan of how the dbt-metricflow Google sheets integration works too and how fast they got it up and running. There are just too many excel users in the world.
I read this and worry that my concerns are real that the data world has been hijacked by people who would rather create governance frameworks and controls, than find (or allow others to find) insights and understanding in data. I think this is in large due to its promotion to ‘the new oil’ and the explosion in roles which are often siloed in the data stack. The increase in status leads to greater scrutiny, and the increase in silos increases the likelihood of errors and mistakes due to lack of insight…which leads again to greater scrutiny.
In governance specifically, I think there is a data lifecycle which starts being produced by business process, when that process is entered into an application, and via ingestions and transformations into reporting, through influencing and into action/decisions. So far regulatory governance approaches I’ve seen only inspect the problem of (a small amount of the ingestion,) transformations and models, but still manages to miss the two key ends of the process. I believe this is one reason why it’s boring - it seems like overhead that is added doesn’t solve the key issue of knowledge gaps in understanding what processes, developer choices, and manager idiosyncrasies help data be productive and valuable
That's an interesting point, and seems probably true? Though not necessarily for nefarious reasons or some sort of intentional capture, but because those things are easier to sell?
Back in the day, we at Mode competed a bunch with Looker. It wasn't an easy sell, because our story was "we help people analyze data and make better decisions," and Looker's story was "we help you set up a governance model so that your dashboards aren't consistent." Though I still believe that the value of a data team is largely in the former story, it's both squishy and hard to achieve. The latter story is both a lot more concrete, and feels like something that can be *completed.* You can build the models; you can check that box. It's a solvable technical problem, rather than a ongoing human problem. And I think there's something much more appealing about that, to both buyers and to people building software.
Ya that’s my impression so far. I’m trying to be open minded... but ya. Oh and absolutely if MS built something extraordinary that they just couldn’t sell to their enterprise customers because it was too big of a change or too different - that certainly doesn’t work. Plus to your point I don’t think they are asking for something completely new and different.
My Microsoft Fabric comments - I have not spent much time - however what I have seen are some existing tools being rebranded as MS Fabric. I took a 5 year break from the Microsoft stack... As I dive back in things feel waay too similar to the SSXSs of old. It feels like the same philosophy with some shiny sprinkled on top. I know there are *some* net new tools. but I struggle to think MS is going to all of a sudden become a leader here....
So it's basically a rebrand + some unified packaging on old stuff?
Microsoft Fabric takes away the boring stuff, and leaves the real work.
Have you used it much? I've heard from people who are optimistic about in theory, but heard very little from people who've actually spent much time doing real work in it.
Ben, I usually provide critical feedback in private, but you strike me as the kind of guy who doesn’t really care 😁
In the paragraph starting “Bluntly” it should say “too closely” instead of “to closely”
How dare you
(thanks; good catch; fixed!)
*nn