Did any of the big ones? I don’t think Looker did; dbt was the consulting thing; Fivetran wasn’t, I don’t believe; Segment was kinda half (I think it was an internal library that got open sourced); I doubt Snowflake was; databricks wasn’t; don’t know about a lot of the ML platform ones.
There is an interesting question in there though. Even if a lot of big analytics projects are internal tools, it seems like there are a lot more that are hyped and *don’t* get big. So it still seems potentially overvalued, even if most successes come from there.
Hmm. I was under the impression that both databricks and Looker had similar origin stories to Mode. I could be wrong about that.
dbt definitely counts though. It is a very common move in consulting to use your billable hours to take the stuff you do all the time and make it into software that you can sell.
Yeah, I genuinely don’t know about Looker. I think LookML might’ve been an idea that came up because of an internal problem, but I don’t think it ever existed as an internal tool.
And true about dbt, though that was my point about consulting. That path makes more sense to me because you make your thing work in a bunch of different contexts. The problem, I think, comes from seeing it do really well in exactly one context, and then trying to extrapolate too far from that.
What’s your opinion on internal tools in the first place? Would you argue they should not exist in the first place and companies should change their practices to work with standard issue solutions? Or do you think they serve there purpose well and are just a cost of doing business?
In general, just the cost of doing business. I'm sure there are times when people over-build them (for some of the reasons I mentioned in the post, people sometimes just want to make stuff). But there are plenty of times when you need a custom solution. It's just that, even if it's useful for you, that custom solution isn't necessarily gonna be useful for many other people.
What many don't understand about data software is that there is an underlying raw material there: data
A good example I recall is talking to Visa. They make X $ on each transaction. Each transaction generates data. While they can deploy and over deploy tons of infrastructure around that data to slice it in infinite types of ways, it does not really effect the X that they make on each transaction. Yes, it might help them make more transactions. But unit economics remain the same. Which means that at a certain point, additional data tooling is a net negative for their EBIDTA, and not worth it.
I think a lot of tech companies chase infinitely for that ideal data universe, but rarely link their buying choices to the unit of data. Instead purchases are linked in headcount. And headcount in data is not very transparent. People can be moved from Data teams to Marketing or Sales.
Eh, I think that’s too pessimistic. Essentially, the argument there is that Visa has a fixed margin they can make on transactions; therefore, nothing about the business can change that much. But that’s clearly not true; Visa can definitely perform badly with those fixed margins, and they can almost certainly perform better.
While data tools and costs might scale with transaction volume, it’s as fraction as much. If Visa can figure out how to increase the number of transactions or average transaction size with that data, they can make a lot more money. Sure, the transaction margin won’t necessarily get better, but the business as a whole makes a lot more money, and the operating margin (which matters a lot more than the transaction margin) can still go up.
Your statement and my statement can both be true, Benn. I am just pointing out that for businesses where Unit Economics matter (most mature businesses), unit margins start to matter a great deal, while the benefits from additional software have diminishing marginal returns.
But it does not mean they should not optimize what they have - for instance, they can still replace their Microstrategy with Mode Analytics ;)
Your comment about startups being created from internal tools at consultancies or professional services teams is so true!
In my both my current role and at a past company, the technology was created in-house at a consulting organization and then spun off into its own product, and in both cases, the adoption has been phenomenal!
One interesting aspect that I've noticed, is that initially we attracted the type of customer that likes to work with consultancies, which makes sense, as the tools were initially built for this specific audience.
There’s probably something interesting in that, about how segregated the Silicon Valley data world is from the consultancies data world (understanding, of course, there’s some overlap and consulting in the former). It seems like both sides view the other with a fair bit of skepticism.
I find it difficult to think of any analytics products in the last 20 years that didn't start as internal tools.
Did any of the big ones? I don’t think Looker did; dbt was the consulting thing; Fivetran wasn’t, I don’t believe; Segment was kinda half (I think it was an internal library that got open sourced); I doubt Snowflake was; databricks wasn’t; don’t know about a lot of the ML platform ones.
There is an interesting question in there though. Even if a lot of big analytics projects are internal tools, it seems like there are a lot more that are hyped and *don’t* get big. So it still seems potentially overvalued, even if most successes come from there.
Hmm. I was under the impression that both databricks and Looker had similar origin stories to Mode. I could be wrong about that.
dbt definitely counts though. It is a very common move in consulting to use your billable hours to take the stuff you do all the time and make it into software that you can sell.
Yeah, I genuinely don’t know about Looker. I think LookML might’ve been an idea that came up because of an internal problem, but I don’t think it ever existed as an internal tool.
And true about dbt, though that was my point about consulting. That path makes more sense to me because you make your thing work in a bunch of different contexts. The problem, I think, comes from seeing it do really well in exactly one context, and then trying to extrapolate too far from that.
What’s your opinion on internal tools in the first place? Would you argue they should not exist in the first place and companies should change their practices to work with standard issue solutions? Or do you think they serve there purpose well and are just a cost of doing business?
In general, just the cost of doing business. I'm sure there are times when people over-build them (for some of the reasons I mentioned in the post, people sometimes just want to make stuff). But there are plenty of times when you need a custom solution. It's just that, even if it's useful for you, that custom solution isn't necessarily gonna be useful for many other people.
I wrote something to a similar effect recently (https://open.substack.com/pub/mirdata/p/data-companies-and-restaurants-so).
What many don't understand about data software is that there is an underlying raw material there: data
A good example I recall is talking to Visa. They make X $ on each transaction. Each transaction generates data. While they can deploy and over deploy tons of infrastructure around that data to slice it in infinite types of ways, it does not really effect the X that they make on each transaction. Yes, it might help them make more transactions. But unit economics remain the same. Which means that at a certain point, additional data tooling is a net negative for their EBIDTA, and not worth it.
I think a lot of tech companies chase infinitely for that ideal data universe, but rarely link their buying choices to the unit of data. Instead purchases are linked in headcount. And headcount in data is not very transparent. People can be moved from Data teams to Marketing or Sales.
Eh, I think that’s too pessimistic. Essentially, the argument there is that Visa has a fixed margin they can make on transactions; therefore, nothing about the business can change that much. But that’s clearly not true; Visa can definitely perform badly with those fixed margins, and they can almost certainly perform better.
While data tools and costs might scale with transaction volume, it’s as fraction as much. If Visa can figure out how to increase the number of transactions or average transaction size with that data, they can make a lot more money. Sure, the transaction margin won’t necessarily get better, but the business as a whole makes a lot more money, and the operating margin (which matters a lot more than the transaction margin) can still go up.
Your statement and my statement can both be true, Benn. I am just pointing out that for businesses where Unit Economics matter (most mature businesses), unit margins start to matter a great deal, while the benefits from additional software have diminishing marginal returns.
But it does not mean they should not optimize what they have - for instance, they can still replace their Microstrategy with Mode Analytics ;)
Hey Benn.
Your comment about startups being created from internal tools at consultancies or professional services teams is so true!
In my both my current role and at a past company, the technology was created in-house at a consulting organization and then spun off into its own product, and in both cases, the adoption has been phenomenal!
One interesting aspect that I've noticed, is that initially we attracted the type of customer that likes to work with consultancies, which makes sense, as the tools were initially built for this specific audience.
There’s probably something interesting in that, about how segregated the Silicon Valley data world is from the consultancies data world (understanding, of course, there’s some overlap and consulting in the former). It seems like both sides view the other with a fair bit of skepticism.