As with Data Contracts and many aspects of the Data Mesh, this model is already in place in many upmarket, larger teams. It's called "IT Chargebacks" which turns the IT/Data team into a mini business within the broader business.
It's used for headcount charging (if marketing wants 10 hours of an analyst time, they'll pay the Head of Data for 10 billable hours), computing resources (if marketing takes up 80% of the Snowflake bill, they cover that, at least in part), licenses (marketing has 50 Looker users so marketing pays IT for 50 Looker seats), and for external product and services vendor RFPs to baseline against.
This is very common in 1000+ headcount companies that run on EBITDA and net operating profit incentives over growth-first incentives.
For further info on how to crawl, walk, run toward this, and as chargeback and showback 101, you can read the following:
I'm loosely familiar with chargebacks, and my understanding is they're more of cost optimization than a useful mechanism for making prioritization decisions (both are useful, sure, but the latter is more academically interesting to me). The amateur economist in me sees the auction/negotiation as the crux of making anything like this work, where the market is a bit more dynamic than how chargebacks typically work.
Well, by assigning cost to an activity like making the nth dashboard or nth DAG or mart or anything, and fixing a dollar value estimate to it, it is then prioritized or even just scrapped altogether if the cost is going to be too high.
True, and I think this model works for just making people be more careful about what they do. But I don't have much faith that IT (or anyone, individually) can set those prices at anywhere near the right levels. I think you've got to have some sort of market for that.
Chargebacks, at least how they were structured at IBM when I was there (i.e. Blue Dollars) had the unfortunate consequence of limiting innovation for potential new products that might otherwise have added a dependency on other offerings. There was no room for experimentation unless an immediate derivative value could be determined. Granted these were sofware products and not data(sets?) and I think you are referring to charging for data professional's time/outputs internally and not necessarily datasets directly, I think there presents a scenario where only work of immediate utility get performed and as a result no incubating projects that could move the needle long term ever get funded. Maybe a good thing maybe not dunno.
I'd see this a different than chargebacks though. Chargebacks feel like a cost control measure, like finance scolding you for doing anything. That's why I'd rather it just be credits, which you have no reason to save. Teams can't trade them in for money later; you use them or lose them. So it's strictly an exercise in prioritization and the data team getting signal on what's most valuable, rather than a way to discourage people from actually using the data team.
In the case where I used quotas I was migrating a bunch of data scientists and processes from on-premise SAS to open source Python on AWS. Previously productivity was measured by model accuracy alone. I switched it to number of models in production with a minimum (a quota). The SAS programmers had to migrate to keep up.
Benn, I would love to do this at my company. I'm sharing an older article on an extreme version of this model shared across the company. I would love to see the stats of the internal app
There are so many fascinating things in this. I wonder if there are things they started charging for, and made free; how did they choose prices; how much of a black market for things are there; i wonder how much this costs to administer; do people actually like it, or is it a huge frustration? Something like this seems to extreme to work, but these sorts of experiments seem so interesting.
Definitely agree the black market data would be fascinating 👀👀👀 and while extreme, a few cultural shifts would help drive people to confront the true value of projects, meetings etc
Nearly every data team I've seen win had to fundamentally change the economics of data in their organization. I've done it by rewarding disparate data teams with talent and headcount, and by setting high quotas. Craig Martell, a Silicon Valley veteran of LinkedIn, Dropbox and Lyft, is doing it now at the largest organization in the world... U.S. Department of Defense. See a good recent interview with Craig here: https://www.youtube.com/watch?v=bfmZ8Iv0uEQ
High quotas for what though? I think that's one of the toughest parts of this; it's tough to set quotas for analytical work (or engineering work, design work, etc) that doesn't feel like paying people for lines of code.
I'll share a story from a team I was previously on. During quarterly planning, while other teams were wrangling over their next Q's commitments, our team would drag everyone into a room together, have product/eng/ux sit at a table, ont table for each subteam that will be working together for the quarter.
Then everyone gets a stack of post-its to write down project/work items, and 13 physical wooden people figures, representing the 13 weeks of a quarter. The teams would have to negotiate and horse trade around with physical tokens how much research, design, eng time could go to each project. It had some shortcomings, but was a surprisingly good system.
I'm sure there are a lot of devils in the details to actually pull something like this off, but it really seems like some sort of scheme like this would work. If nothing else, forcing people to make tradeoffs on a budget seems really valuable. Maybe negotiating with budgets is actually all you need to make it valuable; all the other "It'S aN eCoNoMy" nonsense is mostly worthless distraction.
Yeah the discussion was the whole point of the exercise. Having the fixed tokens to budget with, and everyone else could see where a person's work was going. No thinking "the designers must be free when we need them" because the designers are clearly engaged with other projects for 10wks already and you need 4 and gotta convince other people to give you a token or two... or cut scope
As with Data Contracts and many aspects of the Data Mesh, this model is already in place in many upmarket, larger teams. It's called "IT Chargebacks" which turns the IT/Data team into a mini business within the broader business.
https://journal.uptimeinstitute.com/it-chargeback-drives-efficiency/
It's used for headcount charging (if marketing wants 10 hours of an analyst time, they'll pay the Head of Data for 10 billable hours), computing resources (if marketing takes up 80% of the Snowflake bill, they cover that, at least in part), licenses (marketing has 50 Looker users so marketing pays IT for 50 Looker seats), and for external product and services vendor RFPs to baseline against.
This is very common in 1000+ headcount companies that run on EBITDA and net operating profit incentives over growth-first incentives.
For further info on how to crawl, walk, run toward this, and as chargeback and showback 101, you can read the following:
1) https://www.finops.org/framework/capabilities/chargeback/
2) https://www.softwareone.com/en-us/blog/all-articles/2022/06/20/how-to-establish-a-finops-culture-of-accountability
3) https://www.nicus.com/blog/showback-vs-chargeback-which-should-drive-your-bill-of-it/
I'm loosely familiar with chargebacks, and my understanding is they're more of cost optimization than a useful mechanism for making prioritization decisions (both are useful, sure, but the latter is more academically interesting to me). The amateur economist in me sees the auction/negotiation as the crux of making anything like this work, where the market is a bit more dynamic than how chargebacks typically work.
Well, by assigning cost to an activity like making the nth dashboard or nth DAG or mart or anything, and fixing a dollar value estimate to it, it is then prioritized or even just scrapped altogether if the cost is going to be too high.
But it only works when assigning a cost estimate.
True, and I think this model works for just making people be more careful about what they do. But I don't have much faith that IT (or anyone, individually) can set those prices at anywhere near the right levels. I think you've got to have some sort of market for that.
Chargebacks, at least how they were structured at IBM when I was there (i.e. Blue Dollars) had the unfortunate consequence of limiting innovation for potential new products that might otherwise have added a dependency on other offerings. There was no room for experimentation unless an immediate derivative value could be determined. Granted these were sofware products and not data(sets?) and I think you are referring to charging for data professional's time/outputs internally and not necessarily datasets directly, I think there presents a scenario where only work of immediate utility get performed and as a result no incubating projects that could move the needle long term ever get funded. Maybe a good thing maybe not dunno.
I'd see this a different than chargebacks though. Chargebacks feel like a cost control measure, like finance scolding you for doing anything. That's why I'd rather it just be credits, which you have no reason to save. Teams can't trade them in for money later; you use them or lose them. So it's strictly an exercise in prioritization and the data team getting signal on what's most valuable, rather than a way to discourage people from actually using the data team.
In the case where I used quotas I was migrating a bunch of data scientists and processes from on-premise SAS to open source Python on AWS. Previously productivity was measured by model accuracy alone. I switched it to number of models in production with a minimum (a quota). The SAS programmers had to migrate to keep up.
Ah, interesting. I could see that working for things like that. I suspect it's tough to apply that too generally though.
Benn, I would love to do this at my company. I'm sharing an older article on an extreme version of this model shared across the company. I would love to see the stats of the internal app
https://www.bloomberg.com/news/articles/2019-06-20/charging-employees-for-conference-rooms-helps-disco-boost-profit
There are so many fascinating things in this. I wonder if there are things they started charging for, and made free; how did they choose prices; how much of a black market for things are there; i wonder how much this costs to administer; do people actually like it, or is it a huge frustration? Something like this seems to extreme to work, but these sorts of experiments seem so interesting.
Definitely agree the black market data would be fascinating 👀👀👀 and while extreme, a few cultural shifts would help drive people to confront the true value of projects, meetings etc
Nearly every data team I've seen win had to fundamentally change the economics of data in their organization. I've done it by rewarding disparate data teams with talent and headcount, and by setting high quotas. Craig Martell, a Silicon Valley veteran of LinkedIn, Dropbox and Lyft, is doing it now at the largest organization in the world... U.S. Department of Defense. See a good recent interview with Craig here: https://www.youtube.com/watch?v=bfmZ8Iv0uEQ
High quotas for what though? I think that's one of the toughest parts of this; it's tough to set quotas for analytical work (or engineering work, design work, etc) that doesn't feel like paying people for lines of code.
I'll share a story from a team I was previously on. During quarterly planning, while other teams were wrangling over their next Q's commitments, our team would drag everyone into a room together, have product/eng/ux sit at a table, ont table for each subteam that will be working together for the quarter.
Then everyone gets a stack of post-its to write down project/work items, and 13 physical wooden people figures, representing the 13 weeks of a quarter. The teams would have to negotiate and horse trade around with physical tokens how much research, design, eng time could go to each project. It had some shortcomings, but was a surprisingly good system.
I'm sure there are a lot of devils in the details to actually pull something like this off, but it really seems like some sort of scheme like this would work. If nothing else, forcing people to make tradeoffs on a budget seems really valuable. Maybe negotiating with budgets is actually all you need to make it valuable; all the other "It'S aN eCoNoMy" nonsense is mostly worthless distraction.
Yeah the discussion was the whole point of the exercise. Having the fixed tokens to budget with, and everyone else could see where a person's work was going. No thinking "the designers must be free when we need them" because the designers are clearly engaged with other projects for 10wks already and you need 4 and gotta convince other people to give you a token or two... or cut scope