1) a giant picture of Jared Kushner looking like one of those creatures in Miss Peregrine's Home for Peculiar Children :shudder:
2) Your uncanny ability to post something just as I am thinking about the topic (data culture) and make me question my priors and think different® about the problem at hand. It's really annoying. Thank you.
1.) If it wasn't an email with a length limit, the nice thing to do would've been Jared picture, fifty line breaks, and then start the post. Alas.
2.) Thanks. Though I can't take too much credit. I'm mostly just out here stealing ideas from whatever's the data world's equivalent of Reddit is, and then trying to get credit for them.
Apr 21, 2023·edited Apr 21, 2023Liked by Benn Stancil
Love this! AMITJK - absolutely not! Your writing is thought provoking and forces introspection rather than leaving one disgruntled or upset. Case in point: your mention of Katie and her article. You used it as proof of your point here while also paying respect to her (if she feels disrespected then she should ask AITA about the situation!)
I reflect on your articles a lot after reading them.. it’s almost like the opening arguments for a court case that then unfolds internally over the next week or so.
Thanks! (And that makes me think, I wonder if anyone's ever posted on AITA for posting a question on AITA?)
And I will say, the best thing about writing posts like that is I don't have to argue the rest of the case. I get to make the big showy opening statement at the beginning, and then walk out of the courtroom, and leave the hard work for someone else.
I think the big difference between you and a Jared Kushner is: we know where you live.
Right here. In the trenches. Trying to build a startup, keep up with tech, make a positive dent in the world. If our industry goes belly up, you’ll have to flee the ship with the rest of us. You have skin in the game.
Thank you. That matters more than you can probably imagine.
In my modest capacity, I touched upon a very similar subject and compared it to the Illusion of Asymmetrical Insight.
People believe that their point of view benefits from a greater consensus, and it is the purest expression of that consensus. Stakeholders and Digital Analysts are both prone to this. Domain knowledge does not guarantee that someone is right, but neither is someone armed with data.
The challenge we face in Digital Analytics is stakeholders responding to our recommendations with "so, what?". They believe it lacks alignment with the business needs. I compare this diktat with someone who lost their keys and is looking only under the streetlight.
The idea I put forward is to implement something similar to what Google had for a while: employees spend 20% of their time on passion projects. What would happen when Digital Analysts have 20% of their time to analyse that weird trend that lacked alignment with business needs? A new product or service leading to a new revenue stream? A surprise cost reduction where people were happy with the status quo?
If anybody wants to read more on this, here I am about to shamelessly self-promote my content: Dancing In The Dark
I liked your idea of gamifying the actionable insight delivery. You could pit departments against each other as well. We had something similar at a previous employer about the regular training we had to take. I used go work at a bank. That sort of training is a regulatory requirement.
I was just talking to someone else, and they used a similar analogy that I thought made sense. A lot of analysis is about looking for things in the dark, but you've got this underpowered flashlight and can only see what's very close to you. The work, basically, goes into moving the streetlight.
The other part of that is I think we often think of analysis as looking for keys. It's trying to find that one key insight, that nugget of wisdom, etc. But In reality, I think it's more about trying to figure out what the landscape you're walking on looks like. There is no single view that tells you that; you have to figure it out by just walking around a bunch.
Maybe you allude to that with the journalism paragraph as well. I think the gist of Data team's role is to make a vocal claim; it might be wrong; the world is, after all, very uncertain. But if the Data team does not make a vocal claim about the world - then who will? Middle management won't. Executives often have contradictory objectives within their jobs of leading people. Data people are the only ones who can afford to make vocal claims. But many do not!
At Looker I felt customers sometimes wanted to work with me just because I would tell them how things really are. They could not get this out of their teams, but could get the straight facts out of a vendor. So they paid for the vendor.
I don't think ChatGPT for Data is the answer to this. Making a vocal claim is not something you can get out of sponge filled with facts. It is a matter of having strong, at times, contrarian, opinions. And almost by definition, anything coming out of Machine Learning (Probabilistic) models is not contrarian.
Eh, I'm not sure I'd make it such a proper noun though. I don't think data teams need to go out and make Claims that are bold and contrarian, no more than the news needs to tell stories that are the really dramatic ones. We also just need consistent reminders of what's going on and of how the world works. Customers never use that button; these types of emails work; we're doing really well in Europe. The don't need to be precise stories either, but they're the kinds of things that make us more informed "voters," and I think that 's what we should be chasing.
I think it's valuable and value-adding for folks with an external perspective to come in armed with data, research, and first principles, but they need to come in with humility and an open mind.
For me, this is the essence of the cross-functional knowledge worker team: Folks who are experts (relatively speaking) in their niche, working together to uncover what needs to be done. It's why the marketing-background manager shouldn't decide whether the team should use RMSE or MAPE to measure the model's accuracy, and defer to the junior data scientist with two degrees in statistics. Why we listen to our user researchers explain how users feel about the new feature. And so on.
---
On the suggestion to turn data teams into a different kind of reporting team: I like the idea, but am not sure the data team would be the best owner for doing that work.
The data behind each story is presumably only a subset of the story itself, just like in the real news. Some stories are more about the data (weather reports, election forecasts, covid analyses), but there's many other types: Breaking news, topical buzzers, explainers, and feel-good or crowd-pleasing stories aimed at keeping engagement via entertainment.
I can see the data team being an integral part of this cultural transformation (that's the main thing it is for me), for example providing the tools for the internal comms team to weave data into their stories, and do so properly. Or for article-writing to be a task they have to do, but that lots of other teams also take part it.
If the data team's news reporting is detached from other comms initiatives (big transformations, new product launches, strategy updates, and so on), it seems easy for it to get sidelined or otherwise fall out of place. Just like any other data initiative shouldn't exist to serve a technical strategy detached from the business' goals, this too should be aligned with what the rest of the business is doing and caring about!
That's fair, and I had that thought when I was writing this, that should data teams be the ones who do this? On one hand, they're pretty well positioned to do it; on the other hand, they aren't, obviously, journalists, and that takes some real skill. But that seems like a really tough thing to convince anyone to hire for. Plus, I'd guess that there's a kind of blend for data people, where the "journalism" stuff is more of an extension of their existing work. It's about adding stories to their findings, and sharing other people's findings. Or, to the point of the news quiz, it may not be that the thing they do looks anything like journalism. The point is just to keep people informed. It may be that bombarding people with numbers in slackbots is actually the better way to do this (though I hope not).
I like the idea of having to create (or prompt) some shorter-form content that goes alongside longer pieces - the tweet that goes with the steerco deck, the post that pairs with a new dashboard, and so on.
As I was thinking about what this work would look like, I realised that this is the kind of content creation that generative AI is already good enough for. I met someone a few weeks ago doing this for traditional short-form news reports, which more often than not is basically verbalising data (think weather forecasts, election counts, Covid vaccine uptake etc.)
Would be super interesting to work somewhere that does this, or a version of this! Maybe it’s cause I’m generally curious/nosy about what’s happening around each place I work at but I think it could be quite transformational.
This is all the more relevant if the org has embraced remote working (and relied on word of mouth and in-office interaction to socialise info and knowledge)
And I think some sports sites have actually been doing this for a while with game write ups (and Yahoo did it for fantasy football games too?).
I'm kinda mixed on if that's useful or not though. On one hand, I'm very much with you (and that was sort of the whole point of the post) that these narratives make a huge difference in making things memorable and all that. On the other hand, for some reason, I don't yet have faith that an AI will make that story compelling. Because it's one thing to describe a dashboard in words; it's another thing to make that dashboard relevant by understanding the current narratives around it, what people would find interesting about this dashboard vis a vis that narrative, and so on.
That said, every time I say AI can't do something, a week later, it does. So I'm sure it can here too.
I’ve started adopting the same rule of thumb with comments about AI 😅😅
My gut feeling is that AI will be better at doing this sort of thing than humans, because it’s terribly dull and time-consuming. Of course to do it you’ll need sufficient information encoded somewhere the LLM can access and use as context.
Maybe it’s not just short articles, but also things like infographics? Rolling news tickets? Are we gonna start seeing “ads” with corporate programming next to our office software and dashboards? (Now I want to go make some corporate dystopia art featuring catchy graphics on a sidebar ad inside Outlook or something 😅)
I could see us getting to the point where we have a news corporate news ticker like the ESPN bottom line scrolling across the bottom of dashboards and stuff. Which...I might actually like?
On the ads thing, this is mostly unrelated, but I've got a theory that we're going to eventually get to the point where ad content and creative starts to get personalized. And then, in what starts to feel real dystopian, non-ad content starts injecting personalized ad content into it. So if different people watch a movie on Netflix, the cars the characters drive could be different depending on what they've been searching for.
Like all content personalisation, part of me is fascinated by the idea, and another part dreads and hates it... It seems inevitable though, absent some sort of major shift in politics/regulation. I mean, the Netflix thing is already how sports games personalise stadium ads at the channel-level, so it's not even that far fetched that generative AI is the missing piece of the puzzle to make that scalable on the output side of things (see e.g. https://adscholars.com/wp-content/uploads/2021/07/Football-16-1170x700.png).
Personally I'm less disturbed by the outputs, but worry about the inputs of a personalisation process: Does Netflix being able to tweak its latest TV show to make me want to buy Allbirds rely on them having access to my medical history about how I need orthopaedic insoles? Or does the back-end have privacy safeguards baked into its design? We're doing some work on something related to this at work, helping advertisers verify the audiences they market to without them actually getting their hands on the data required to do it, and it makes me feel a lot less icky about it (in the way most advertising work would)
Living in the Middle East and being opinionated about various things - I get it.
a couple of thoughts :
1. You can't understand the complexity of something like the Middle East or go language programming without having actually done it and lived in the environment for a while. Having said that - after you lived in Tel Aviv and then Jerusalem for a few years - you need to be able to go up to 20,000 feet and detach yourself from the feeds and neediness for likes and approbation.
In other words you need both first-hand context and detachment. Which is very hard.
2. We noticed that our clinical data users pretty much ignore the dashboards and get addicted to alerts. I know we can do better dashboards - but I think that the reason is context and timeliness.
By the time the events roll up thru a ELT process into a dashboard - it's too late.
Alerts have a narrow context and time window - 'This patient did not do that in the past 48 hours'
Tying this back to the Kushner metaphor - when you read some books and watch some youtube videos you don't have the context of having lived there or the timeliness of having been in Tel aviv next to Palestinian terrorist shooting people in a bar.
Thanks! And that's a good point - on the ground (in business or elsewhere, I'd imagine) it's probably easy to start seeing patterns that aren't really there, and then a bunch of confirmation biases making you believe it even more. But that's a really hard balance to strike, of both listening to "the numbers" while also respecting and paying attention to the lived experiences.
One way that you're clearly not Jared Kushner is your opinions are known and they are public. If you're wrong, we can go back through your history of published work and tell that you were wrong. Who heard any of Jared's opinions? In fact, let me restate that: who heard what Jared Kushner even sounds like?
There are two things i hate about this post:
1) a giant picture of Jared Kushner looking like one of those creatures in Miss Peregrine's Home for Peculiar Children :shudder:
2) Your uncanny ability to post something just as I am thinking about the topic (data culture) and make me question my priors and think different® about the problem at hand. It's really annoying. Thank you.
1.) If it wasn't an email with a length limit, the nice thing to do would've been Jared picture, fifty line breaks, and then start the post. Alas.
2.) Thanks. Though I can't take too much credit. I'm mostly just out here stealing ideas from whatever's the data world's equivalent of Reddit is, and then trying to get credit for them.
Love this! AMITJK - absolutely not! Your writing is thought provoking and forces introspection rather than leaving one disgruntled or upset. Case in point: your mention of Katie and her article. You used it as proof of your point here while also paying respect to her (if she feels disrespected then she should ask AITA about the situation!)
I reflect on your articles a lot after reading them.. it’s almost like the opening arguments for a court case that then unfolds internally over the next week or so.
Thanks! (And that makes me think, I wonder if anyone's ever posted on AITA for posting a question on AITA?)
And I will say, the best thing about writing posts like that is I don't have to argue the rest of the case. I get to make the big showy opening statement at the beginning, and then walk out of the courtroom, and leave the hard work for someone else.
I think the big difference between you and a Jared Kushner is: we know where you live.
Right here. In the trenches. Trying to build a startup, keep up with tech, make a positive dent in the world. If our industry goes belly up, you’ll have to flee the ship with the rest of us. You have skin in the game.
Thank you. That matters more than you can probably imagine.
I appreciate that, but...wait until I sell my soul to marry a Koch daughter, spin it into an appointment as an ambassador, and sell everyone's data to MBS for a ten billion dollars of oil contracts and a chance to touch The Orb. https://www.nytimes.com/2017/05/22/world/middleeast/trump-glowing-orb-saudi.html
Well, sure. At that point I’ll despise you. Until then, I have no choice but to admire you for your moral courage and vulnerable honesty. :-)
In my modest capacity, I touched upon a very similar subject and compared it to the Illusion of Asymmetrical Insight.
People believe that their point of view benefits from a greater consensus, and it is the purest expression of that consensus. Stakeholders and Digital Analysts are both prone to this. Domain knowledge does not guarantee that someone is right, but neither is someone armed with data.
The challenge we face in Digital Analytics is stakeholders responding to our recommendations with "so, what?". They believe it lacks alignment with the business needs. I compare this diktat with someone who lost their keys and is looking only under the streetlight.
The idea I put forward is to implement something similar to what Google had for a while: employees spend 20% of their time on passion projects. What would happen when Digital Analysts have 20% of their time to analyse that weird trend that lacked alignment with business needs? A new product or service leading to a new revenue stream? A surprise cost reduction where people were happy with the status quo?
If anybody wants to read more on this, here I am about to shamelessly self-promote my content: Dancing In The Dark
https://www.linkedin.com/pulse/dancing-dark-alban-g%C3%A9r%C3%B4me?utm_source=share&utm_medium=member_android&utm_campaign=share_via
I liked your idea of gamifying the actionable insight delivery. You could pit departments against each other as well. We had something similar at a previous employer about the regular training we had to take. I used go work at a bank. That sort of training is a regulatory requirement.
I was just talking to someone else, and they used a similar analogy that I thought made sense. A lot of analysis is about looking for things in the dark, but you've got this underpowered flashlight and can only see what's very close to you. The work, basically, goes into moving the streetlight.
The other part of that is I think we often think of analysis as looking for keys. It's trying to find that one key insight, that nugget of wisdom, etc. But In reality, I think it's more about trying to figure out what the landscape you're walking on looks like. There is no single view that tells you that; you have to figure it out by just walking around a bunch.
Maybe you allude to that with the journalism paragraph as well. I think the gist of Data team's role is to make a vocal claim; it might be wrong; the world is, after all, very uncertain. But if the Data team does not make a vocal claim about the world - then who will? Middle management won't. Executives often have contradictory objectives within their jobs of leading people. Data people are the only ones who can afford to make vocal claims. But many do not!
At Looker I felt customers sometimes wanted to work with me just because I would tell them how things really are. They could not get this out of their teams, but could get the straight facts out of a vendor. So they paid for the vendor.
I don't think ChatGPT for Data is the answer to this. Making a vocal claim is not something you can get out of sponge filled with facts. It is a matter of having strong, at times, contrarian, opinions. And almost by definition, anything coming out of Machine Learning (Probabilistic) models is not contrarian.
Eh, I'm not sure I'd make it such a proper noun though. I don't think data teams need to go out and make Claims that are bold and contrarian, no more than the news needs to tell stories that are the really dramatic ones. We also just need consistent reminders of what's going on and of how the world works. Customers never use that button; these types of emails work; we're doing really well in Europe. The don't need to be precise stories either, but they're the kinds of things that make us more informed "voters," and I think that 's what we should be chasing.
"at times", contrarian. The key words were "having opinions". But yeah, mostly agree with you.
the data quiz is a delight of a concept. would be curious if anyone has tried / succeeded
I don't know anyone who's done it directly, but Katie has tried something kinda like it: https://twitter.com/imightbemary/status/1650147927258718208
Benn, really interesting essay, as usual.
I think it's valuable and value-adding for folks with an external perspective to come in armed with data, research, and first principles, but they need to come in with humility and an open mind.
For me, this is the essence of the cross-functional knowledge worker team: Folks who are experts (relatively speaking) in their niche, working together to uncover what needs to be done. It's why the marketing-background manager shouldn't decide whether the team should use RMSE or MAPE to measure the model's accuracy, and defer to the junior data scientist with two degrees in statistics. Why we listen to our user researchers explain how users feel about the new feature. And so on.
---
On the suggestion to turn data teams into a different kind of reporting team: I like the idea, but am not sure the data team would be the best owner for doing that work.
The data behind each story is presumably only a subset of the story itself, just like in the real news. Some stories are more about the data (weather reports, election forecasts, covid analyses), but there's many other types: Breaking news, topical buzzers, explainers, and feel-good or crowd-pleasing stories aimed at keeping engagement via entertainment.
I can see the data team being an integral part of this cultural transformation (that's the main thing it is for me), for example providing the tools for the internal comms team to weave data into their stories, and do so properly. Or for article-writing to be a task they have to do, but that lots of other teams also take part it.
If the data team's news reporting is detached from other comms initiatives (big transformations, new product launches, strategy updates, and so on), it seems easy for it to get sidelined or otherwise fall out of place. Just like any other data initiative shouldn't exist to serve a technical strategy detached from the business' goals, this too should be aligned with what the rest of the business is doing and caring about!
That's fair, and I had that thought when I was writing this, that should data teams be the ones who do this? On one hand, they're pretty well positioned to do it; on the other hand, they aren't, obviously, journalists, and that takes some real skill. But that seems like a really tough thing to convince anyone to hire for. Plus, I'd guess that there's a kind of blend for data people, where the "journalism" stuff is more of an extension of their existing work. It's about adding stories to their findings, and sharing other people's findings. Or, to the point of the news quiz, it may not be that the thing they do looks anything like journalism. The point is just to keep people informed. It may be that bombarding people with numbers in slackbots is actually the better way to do this (though I hope not).
I like the idea of having to create (or prompt) some shorter-form content that goes alongside longer pieces - the tweet that goes with the steerco deck, the post that pairs with a new dashboard, and so on.
As I was thinking about what this work would look like, I realised that this is the kind of content creation that generative AI is already good enough for. I met someone a few weeks ago doing this for traditional short-form news reports, which more often than not is basically verbalising data (think weather forecasts, election counts, Covid vaccine uptake etc.)
Would be super interesting to work somewhere that does this, or a version of this! Maybe it’s cause I’m generally curious/nosy about what’s happening around each place I work at but I think it could be quite transformational.
This is all the more relevant if the org has embraced remote working (and relied on word of mouth and in-office interaction to socialise info and knowledge)
So Tableau and PowerBI can do things like this now, I think:
- https://www.tableau.com/blog/tableau-and-narrative-science-make-data-more-accessible
- https://learn.microsoft.com/en-us/power-bi/create-reports/desktop-insights
And I think some sports sites have actually been doing this for a while with game write ups (and Yahoo did it for fantasy football games too?).
I'm kinda mixed on if that's useful or not though. On one hand, I'm very much with you (and that was sort of the whole point of the post) that these narratives make a huge difference in making things memorable and all that. On the other hand, for some reason, I don't yet have faith that an AI will make that story compelling. Because it's one thing to describe a dashboard in words; it's another thing to make that dashboard relevant by understanding the current narratives around it, what people would find interesting about this dashboard vis a vis that narrative, and so on.
That said, every time I say AI can't do something, a week later, it does. So I'm sure it can here too.
I’ve started adopting the same rule of thumb with comments about AI 😅😅
My gut feeling is that AI will be better at doing this sort of thing than humans, because it’s terribly dull and time-consuming. Of course to do it you’ll need sufficient information encoded somewhere the LLM can access and use as context.
Maybe it’s not just short articles, but also things like infographics? Rolling news tickets? Are we gonna start seeing “ads” with corporate programming next to our office software and dashboards? (Now I want to go make some corporate dystopia art featuring catchy graphics on a sidebar ad inside Outlook or something 😅)
I could see us getting to the point where we have a news corporate news ticker like the ESPN bottom line scrolling across the bottom of dashboards and stuff. Which...I might actually like?
On the ads thing, this is mostly unrelated, but I've got a theory that we're going to eventually get to the point where ad content and creative starts to get personalized. And then, in what starts to feel real dystopian, non-ad content starts injecting personalized ad content into it. So if different people watch a movie on Netflix, the cars the characters drive could be different depending on what they've been searching for.
Like all content personalisation, part of me is fascinated by the idea, and another part dreads and hates it... It seems inevitable though, absent some sort of major shift in politics/regulation. I mean, the Netflix thing is already how sports games personalise stadium ads at the channel-level, so it's not even that far fetched that generative AI is the missing piece of the puzzle to make that scalable on the output side of things (see e.g. https://adscholars.com/wp-content/uploads/2021/07/Football-16-1170x700.png).
Personally I'm less disturbed by the outputs, but worry about the inputs of a personalisation process: Does Netflix being able to tweak its latest TV show to make me want to buy Allbirds rely on them having access to my medical history about how I need orthopaedic insoles? Or does the back-end have privacy safeguards baked into its design? We're doing some work on something related to this at work, helping advertisers verify the audiences they market to without them actually getting their hands on the data required to do it, and it makes me feel a lot less icky about it (in the way most advertising work would)
Plot twist: enterprise Substack is the product and data teams will now be paid for via annual subscription
you joke but https://benn.substack.com/p/datas-invisible-hand#:~:text=Data%20teams%20could%20charge%20subscription%20fees%20to%20keep%20their%20time%20on%20retainer%2C%20in%20case%20unexpected%20questions%20come%20up%20in%20the%20normal%20course%20of%20business
Great starting hook -
Living in the Middle East and being opinionated about various things - I get it.
a couple of thoughts :
1. You can't understand the complexity of something like the Middle East or go language programming without having actually done it and lived in the environment for a while. Having said that - after you lived in Tel Aviv and then Jerusalem for a few years - you need to be able to go up to 20,000 feet and detach yourself from the feeds and neediness for likes and approbation.
In other words you need both first-hand context and detachment. Which is very hard.
2. We noticed that our clinical data users pretty much ignore the dashboards and get addicted to alerts. I know we can do better dashboards - but I think that the reason is context and timeliness.
By the time the events roll up thru a ELT process into a dashboard - it's too late.
Alerts have a narrow context and time window - 'This patient did not do that in the past 48 hours'
Tying this back to the Kushner metaphor - when you read some books and watch some youtube videos you don't have the context of having lived there or the timeliness of having been in Tel aviv next to Palestinian terrorist shooting people in a bar.
Thanks! And that's a good point - on the ground (in business or elsewhere, I'd imagine) it's probably easy to start seeing patterns that aren't really there, and then a bunch of confirmation biases making you believe it even more. But that's a really hard balance to strike, of both listening to "the numbers" while also respecting and paying attention to the lived experiences.
Who are you, one other reader of Benn's blog from Tel Aviv. We should meet
One way that you're clearly not Jared Kushner is your opinions are known and they are public. If you're wrong, we can go back through your history of published work and tell that you were wrong. Who heard any of Jared's opinions? In fact, let me restate that: who heard what Jared Kushner even sounds like?
Somehow, the first time I heard him talk, it was both very surprising and exactly what you would expect.
And whatever you do, please don't go back and reread old things; I can't imagine that would end well.
This is awesome. Great points plus you worked in Good Will Hunting and took shots at Jarad Kusher.
Sometimes, you gotta just play the hits.
I'm Danny Lieberman - sure. You can see my profile on LI