To me, this dynamic is the single most determinative fact on this topic. If I had some magical way to quantify the output of data analysts and analytics engineers, my gut instinct is that it would line up directly with your charts in that post. Which is to say: the median data analyst provides less value but the max data analyst provides dramatically _more value_ than an analytics engineer. The distributions look very different.
I know some of those top-end-of-the-distribution analysts and I could imagine easily making the argument that they should get paid $500k+ if they're working in a domain area that rewards it. And the thing that is special about those folks is, as you say, not their "technical" skills...it's their domain knowledge and "soft skills".
Sometimes traditional career-ladder-focused compensation practices don't do a good job of accurately valuing impact that is distributed in this way.
That said, I think this only works if there are more companies that would be willing to hire those very top-end analysts. Big companies might have roles like that but most don't, so the this structure (which feels exactly right to me) is still mostly theoretical.
Preach it, brother! That’s why I’m pushing the term “data therapist” :-) The hardest work of all is helping people productively articulate what they truly want…
I’m pushing data detective. In large orgs the challenge is first asking the right questions, then finding the evidence and putting together the right narrative. The large number of engagements to find out what answers exist in the lake/warehouse/source system/spreadsheet feels like being a film noir detective. Probably should buy myself a coat and hat
I think this sort of approach is directionally good, but really vague. In my experience, five or six years ago, data folks tended to see themselves as rational, empirical truth tellers. Now, I think there's been a big shift towards ideals of being empathetic, human, etc, without a ton of specification about what that actually means (or, more cynically, saying it without actually living up to it).
Sounds like a market opportunity to create rigorous standards around how to be productively empathic. Maybe that’ll be our 10x improvement over financial metrics. :-)
There's definitely a technical skill hierarchy in the data world's zeitgeist: SWE, Data > Data Engineer (platform) > Analytics Engineer > Data Analyst / Scientist / BI Developer. This is reflected both in salaries and community prestige. Analytics work is "easy" and "boring"; SQL development is "simple"; data engineers "use simplifying frameworks and tooling"; and SWEs, the real heroes, are writing complex distributed backends using cutting-edge algos and their enormous brains.
Of course this is nonsense; you can do some incredibly complex analyses and write some very simplistic event emission; then again, if you suggest that you can teach someone to write Kafka consumers more easily than you can teach someone to create a reliable forecast, a lot of people would bite your head off. If an analyst identifies $100M of revenue using a model an AE built that comes from data a DE landed and that the SWEs ultimately surfaced through the product, don't they all deserve credit for their respective parts in the value chain?
Very good post, thanks for sharing. Having bounced around in a bunch of different jobs at Mode, I can definitely say it's all hard (though maybe that says more about me than the job).
Though sports analogies are a bit overplayed, it feels like there's some lesson to be drawn from football teams here. It seems like it in football, the clear difference in people's ability to play different positions (WRs can't be offensive linemen, and vice versa) makes it easier for folks to all celebrate their roles. Within companies, those differences are probably there; they're just much harder to see. I wonder if there are ways to make them more apparent.
I end up interviewing lots of people who are very technical, a fair number of people who can write and communicate well, and very few people who embody both. I've found it's very hard to get HR to value that though.
And very hard to screen for, I'd argue. Their resume says "Experience in PySpark" or whatever is a much easier flag than "this person is going to be going to make persuasive presentations about nuanced issues to opinionated execs."
Yeah, difficult to get from a resume (and almost impossible from just glancing at a resume). When I worked in non-profit data, I used to ask first round candidates how they thought the industry used data poorly. It served as a useful screen for people that were a bit thoughtful/had thought strategically about data and the industry as a whole. I imagine you could do the same as a text prompt at the application window.
Hah, if they spent too much time on a well written cover letter, that tells me they aren't thinking strategically at all.
Joke aside, I have yet to read a cover letter that swayed me on a candidate. I devoted a lot of time to writing them when I was first trying to change industries, but networking/cold emailing hiring managers (to bypass the recruiter screen) seemed to be more effective. I'm open to it though.
Same, I usually opened them and never really read them. Though every once in a while, something would stand out when I was glancing at it that would make me stop and read it. And I think I might've moved a couple candidates ahead to the next round based on that? I think there were a couple cases where I thought, "This is good enough that it's worth taking a shot on a phone screen." No idea if they made it further than that, though.
I wonder if another dynamic is that the "analyst" title can span a broader range of capabilities than "analytics engineer"? The base level capability of an analytics engineer is already fairly high because there are certain things you need to be able to do to do your job. Whereas the base level of someone calling themselves an analyst is possibly lower, because the role is broader and more amorphous. So being able to make a pie chart in excel and have an opinion about what it means is probably enough to call yourself an analyst?
I don't at all mean to be disparaging. On the other end there are analysts absolutely doing very sophisticated work on the technical side and the analytical side and the business/product side and the people side. If anything, between analyst and analyst engineer, I'm an analyst.
I think there is maybe a similar but opposite problem with the data scientist title, because like analysts they are inter-disciplinary roles and need expertise in a pretty broad range of skills. Maybe the breadth of the role makes it too difficult to quantify?
So on the data scientist side (maybe because it started with PhDs? Don't know?) the breadth made people label it as fancy, whereas on the analyst side the breadth made people label it as unsophisticated / untechnical (also perhaps because it is a more female dominated field and that means it's easier, right?)
I think there's something to that. My current view of this is that analysts and analytics engineers are probably best treated more or less the same, like a different side of the same role. That said, I'd agree that there probably aren't (yet, anyway) entry level analytics engineering positions.
I'm not sure what happens at the top. It seems to me that you could climb up the latter on both, where you're focused on organization-wide impact in both cases.
It takes constant learning and skills improvement to be a Systems/Data/Software Engineer. Platforms, frameworks, and the technology ecosystem are constantly changing. Most of the time, this learning effort is done outside of the regular work schedule at the sacrifice of a personal life because there just isn't enough time in a day to do it all. The amount of information and the toolset any one of these positions requires is overwhelming.
Could you not say that about other jobs though? Being an HR leader, for instance, requires you to constantly stay on top of changing laws, cultural norms, employee expectations, and the opinions (not to mention feelings) of potentially hundreds and thousands of employees. It's less visible, perhaps, but I'd argue just as overwhelming.
Sure! Regardless of your title or industry, some individuals are going to do the bare minimum to stay employed. They do the same thing day in and day out by some script created either internally in their own brain or handed to them. In my opinion, these individuals could be replaced with automation. They may be titled engineers, but they are really technicians. They did the initial work and received the training to get the job and they are happy to live in the moment and do what they are told. Other individuals with the same title are creating that automation while continuously learning and recommending ways to improve "the system".
Some positions may have more to learn and sort through than others. In the HR example, laws are handed down, sure they need to be interpreted, but in IT there is no concrete best way to do X. You have to learn all the ways to do X and determine which is the best for your situation. If you are not learning ways to improve the system, the system will eventually fail and you should not call yourself an engineer. The HR feelings aspect and cultural studies in your example could be considered Human Engineering :) and should be compensated accordingly.
There are data analysts that simply prepare the data, present the data, and give a cursory explanation of what is obvious. They use the same query language they've used for the last 20 years and tools similar to the tools they have used in the past. Other analysts may tackle larger, more complex datasets and need a little engineering insight to dig in to investigate the data or even recommend changes to how the data is engineered and delivered while also generating ground breaking insights and business recommendations. These two groups of people should not be paid the same. With that said, I'm a bit on the fence about what happens when low effort accidentally stumbles upon a ground breaking insight. I guess it is similar to winning the lottery. It is the soft skills and the technical skills, that set these analysts apart.
Back to the engineer vs analyst argument. How much change and adaption is needed between the engineer and the analyst? How drastically does the engineering toolset evolve vs the toolset used by the analyst? As with every other question in the universe, "It depends."
It boils down to the fact that nothing is absolute. Even if we say that people should be paid based on their effort and value produced, how is that measured? Do I have to record every meeting I'm in where I advised a team of a best practice or recommended a solution to a problem so I can prove my worth? Do I record every mentoring session? How about paying me for how quickly I helped my new manager onboard? Most of what I do isn't even in my job description, but it needs to be done.
Job titles are just labels. People should be paid for what they bring to an organization. Here's an idea, make each manager's pay based on a percentage of their direct reports pay. For a manager to make more, their reports have to be paid more and let the managers fight for fair pay. I have other things that need to get done :)
This is wading into areas that I’m sure people much smarter than me have studied and thought a lot about, but I’d be interested in what’s been tried on variable pay for roles that don’t traditionally have it. As you said, people in the same roles and with the same titles can produce a wide range of outputs, and it’s often very difficult to compensate those folks according to that.
My general assumption is that trying to do so is a bad idea, because as you say, it’s very hard to measure in an even remotely objective way. I have to imagine that people have tried different things though. It’d be interesting to know what came out of those experiments. I wonder if anything got close to working, or if it was all just a disaster.
Jun 1, 2022·edited Jun 1, 2022Liked by Benn Stancil
We had a similar ongoing observation in my software development firm a few years ago. Our incoming business analysts (or product owners, or product managers, etc) generally expected less pay than software engineers. Market data provided by recruiting agencies mirrored that. But by all our measures—customer account growth, longevity, profitability—the ones with strong BAs in the lead were clear winners.
Tristan was spot on too—the good ones are far, far better than the average. And maybe that is a factor in this. I can claim to be a capable “soft skill” person, and that is harder to disprove than some baseline of tech skills
Agreed, and there's not really even a model for paying people who are valuable well beyond the normal band. Sales teams do it with quotas and bonuses, but it's pretty rare for non-GTM positions to have that. I guess you get there with discretionary bonuses, though that feels pretty tricky to administer. (Facebook does this though, and I've heard of cases when some employees have gotten 100%+ of their salary in a bonus after a really big year.)
Here in Australia there is a divide. Engineers are likely to have a definite qualification, data scientists probably do (but do not assume too much) but for someone called an 'analyst' there is a culture in organisations that they do not need a qualification and just need to know some SQL. So we have a lot of people doing analytics who might find statistical thinking a challenge, and this overall detracts from the perceived value of the work of analysts. Engineers, though, command a price because they presumable could be doing something else of value.
Interesting - I've never thought about it from that angle. There's a lot of commentary about how analysts are hard to measure and their skills are hard to assess. This feels like a corollary to that that I've never considered, which is that analysts are also less consistent in what they do. Since the general perception of an analyst (which probably drives their pay) is a blend of of a lot of skillsets, the composite mash up of an analyst is pretty vague and potentially uninspiring.
Didn't you write a post about the work done by data analysts and compare it to honeydew? Ah yes, found it:
https://benn.substack.com/p/third-rail
To me, this dynamic is the single most determinative fact on this topic. If I had some magical way to quantify the output of data analysts and analytics engineers, my gut instinct is that it would line up directly with your charts in that post. Which is to say: the median data analyst provides less value but the max data analyst provides dramatically _more value_ than an analytics engineer. The distributions look very different.
I know some of those top-end-of-the-distribution analysts and I could imagine easily making the argument that they should get paid $500k+ if they're working in a domain area that rewards it. And the thing that is special about those folks is, as you say, not their "technical" skills...it's their domain knowledge and "soft skills".
Sometimes traditional career-ladder-focused compensation practices don't do a good job of accurately valuing impact that is distributed in this way.
I think this is right. Abhi made a similar point on Twitter, where analytics engineers are basically a middle-level role, and he's structured his team this way: https://twitter.com/_abhisivasailam/status/1530269554353397760
That said, I think this only works if there are more companies that would be willing to hire those very top-end analysts. Big companies might have roles like that but most don't, so the this structure (which feels exactly right to me) is still mostly theoretical.
Tristan, analysts are not a cult.
You are safe.
No need for this, Lauren. I value the conversations here; let's keep them substantive.
Preach it, brother! That’s why I’m pushing the term “data therapist” :-) The hardest work of all is helping people productively articulate what they truly want…
I’m pushing data detective. In large orgs the challenge is first asking the right questions, then finding the evidence and putting together the right narrative. The large number of engagements to find out what answers exist in the lake/warehouse/source system/spreadsheet feels like being a film noir detective. Probably should buy myself a coat and hat
I'm a big fan of this description, because it makes clear we're fairly ignorant on our own, and we can only solve problems by talking to witnesses.
https://benn.substack.com/p/service-pressure?s=w#:~:text=Be%20a%20detective%2C%20not%20a%20consultant
I like this new term/role - Data Therapist :)
I tried (and failed) to push the idea at Coalesce last year. https://ihack.us/2021/05/13/dbt-as-the-couch-for-organizational-therapy/ Wrestling with how/whether to try again this year. Jillian suggested “Analytics Anonymous”!
I think this sort of approach is directionally good, but really vague. In my experience, five or six years ago, data folks tended to see themselves as rational, empirical truth tellers. Now, I think there's been a big shift towards ideals of being empathetic, human, etc, without a ton of specification about what that actually means (or, more cynically, saying it without actually living up to it).
Sounds like a market opportunity to create rigorous standards around how to be productively empathic. Maybe that’ll be our 10x improvement over financial metrics. :-)
As usual, the footnotes are (nearly) as good as the post itself. Thanks Benn.
Charity Majors has a great article that touches on your "more technical" point: https://charity.wtf/2022/01/20/how-engineering-driven-leads-to-engineering-supremacy/.
There's definitely a technical skill hierarchy in the data world's zeitgeist: SWE, Data > Data Engineer (platform) > Analytics Engineer > Data Analyst / Scientist / BI Developer. This is reflected both in salaries and community prestige. Analytics work is "easy" and "boring"; SQL development is "simple"; data engineers "use simplifying frameworks and tooling"; and SWEs, the real heroes, are writing complex distributed backends using cutting-edge algos and their enormous brains.
Of course this is nonsense; you can do some incredibly complex analyses and write some very simplistic event emission; then again, if you suggest that you can teach someone to write Kafka consumers more easily than you can teach someone to create a reliable forecast, a lot of people would bite your head off. If an analyst identifies $100M of revenue using a model an AE built that comes from data a DE landed and that the SWEs ultimately surfaced through the product, don't they all deserve credit for their respective parts in the value chain?
Very good post, thanks for sharing. Having bounced around in a bunch of different jobs at Mode, I can definitely say it's all hard (though maybe that says more about me than the job).
Though sports analogies are a bit overplayed, it feels like there's some lesson to be drawn from football teams here. It seems like it in football, the clear difference in people's ability to play different positions (WRs can't be offensive linemen, and vice versa) makes it easier for folks to all celebrate their roles. Within companies, those differences are probably there; they're just much harder to see. I wonder if there are ways to make them more apparent.
I end up interviewing lots of people who are very technical, a fair number of people who can write and communicate well, and very few people who embody both. I've found it's very hard to get HR to value that though.
And very hard to screen for, I'd argue. Their resume says "Experience in PySpark" or whatever is a much easier flag than "this person is going to be going to make persuasive presentations about nuanced issues to opinionated execs."
Yeah, difficult to get from a resume (and almost impossible from just glancing at a resume). When I worked in non-profit data, I used to ask first round candidates how they thought the industry used data poorly. It served as a useful screen for people that were a bit thoughtful/had thought strategically about data and the industry as a whole. I imagine you could do the same as a text prompt at the application window.
The weird thing is, I bet that cover letters are actually a decent screen for that? And the most ignored thing in applications are cover letters.
Hah, if they spent too much time on a well written cover letter, that tells me they aren't thinking strategically at all.
Joke aside, I have yet to read a cover letter that swayed me on a candidate. I devoted a lot of time to writing them when I was first trying to change industries, but networking/cold emailing hiring managers (to bypass the recruiter screen) seemed to be more effective. I'm open to it though.
Same, I usually opened them and never really read them. Though every once in a while, something would stand out when I was glancing at it that would make me stop and read it. And I think I might've moved a couple candidates ahead to the next round based on that? I think there were a couple cases where I thought, "This is good enough that it's worth taking a shot on a phone screen." No idea if they made it further than that, though.
I wonder if another dynamic is that the "analyst" title can span a broader range of capabilities than "analytics engineer"? The base level capability of an analytics engineer is already fairly high because there are certain things you need to be able to do to do your job. Whereas the base level of someone calling themselves an analyst is possibly lower, because the role is broader and more amorphous. So being able to make a pie chart in excel and have an opinion about what it means is probably enough to call yourself an analyst?
I don't at all mean to be disparaging. On the other end there are analysts absolutely doing very sophisticated work on the technical side and the analytical side and the business/product side and the people side. If anything, between analyst and analyst engineer, I'm an analyst.
I think there is maybe a similar but opposite problem with the data scientist title, because like analysts they are inter-disciplinary roles and need expertise in a pretty broad range of skills. Maybe the breadth of the role makes it too difficult to quantify?
So on the data scientist side (maybe because it started with PhDs? Don't know?) the breadth made people label it as fancy, whereas on the analyst side the breadth made people label it as unsophisticated / untechnical (also perhaps because it is a more female dominated field and that means it's easier, right?)
Just some musings!
I think there's something to that. My current view of this is that analysts and analytics engineers are probably best treated more or less the same, like a different side of the same role. That said, I'd agree that there probably aren't (yet, anyway) entry level analytics engineering positions.
I'm not sure what happens at the top. It seems to me that you could climb up the latter on both, where you're focused on organization-wide impact in both cases.
It takes constant learning and skills improvement to be a Systems/Data/Software Engineer. Platforms, frameworks, and the technology ecosystem are constantly changing. Most of the time, this learning effort is done outside of the regular work schedule at the sacrifice of a personal life because there just isn't enough time in a day to do it all. The amount of information and the toolset any one of these positions requires is overwhelming.
Could you not say that about other jobs though? Being an HR leader, for instance, requires you to constantly stay on top of changing laws, cultural norms, employee expectations, and the opinions (not to mention feelings) of potentially hundreds and thousands of employees. It's less visible, perhaps, but I'd argue just as overwhelming.
Sure! Regardless of your title or industry, some individuals are going to do the bare minimum to stay employed. They do the same thing day in and day out by some script created either internally in their own brain or handed to them. In my opinion, these individuals could be replaced with automation. They may be titled engineers, but they are really technicians. They did the initial work and received the training to get the job and they are happy to live in the moment and do what they are told. Other individuals with the same title are creating that automation while continuously learning and recommending ways to improve "the system".
Some positions may have more to learn and sort through than others. In the HR example, laws are handed down, sure they need to be interpreted, but in IT there is no concrete best way to do X. You have to learn all the ways to do X and determine which is the best for your situation. If you are not learning ways to improve the system, the system will eventually fail and you should not call yourself an engineer. The HR feelings aspect and cultural studies in your example could be considered Human Engineering :) and should be compensated accordingly.
There are data analysts that simply prepare the data, present the data, and give a cursory explanation of what is obvious. They use the same query language they've used for the last 20 years and tools similar to the tools they have used in the past. Other analysts may tackle larger, more complex datasets and need a little engineering insight to dig in to investigate the data or even recommend changes to how the data is engineered and delivered while also generating ground breaking insights and business recommendations. These two groups of people should not be paid the same. With that said, I'm a bit on the fence about what happens when low effort accidentally stumbles upon a ground breaking insight. I guess it is similar to winning the lottery. It is the soft skills and the technical skills, that set these analysts apart.
Back to the engineer vs analyst argument. How much change and adaption is needed between the engineer and the analyst? How drastically does the engineering toolset evolve vs the toolset used by the analyst? As with every other question in the universe, "It depends."
It boils down to the fact that nothing is absolute. Even if we say that people should be paid based on their effort and value produced, how is that measured? Do I have to record every meeting I'm in where I advised a team of a best practice or recommended a solution to a problem so I can prove my worth? Do I record every mentoring session? How about paying me for how quickly I helped my new manager onboard? Most of what I do isn't even in my job description, but it needs to be done.
Job titles are just labels. People should be paid for what they bring to an organization. Here's an idea, make each manager's pay based on a percentage of their direct reports pay. For a manager to make more, their reports have to be paid more and let the managers fight for fair pay. I have other things that need to get done :)
This is wading into areas that I’m sure people much smarter than me have studied and thought a lot about, but I’d be interested in what’s been tried on variable pay for roles that don’t traditionally have it. As you said, people in the same roles and with the same titles can produce a wide range of outputs, and it’s often very difficult to compensate those folks according to that.
My general assumption is that trying to do so is a bad idea, because as you say, it’s very hard to measure in an even remotely objective way. I have to imagine that people have tried different things though. It’d be interesting to know what came out of those experiments. I wonder if anything got close to working, or if it was all just a disaster.
We had a similar ongoing observation in my software development firm a few years ago. Our incoming business analysts (or product owners, or product managers, etc) generally expected less pay than software engineers. Market data provided by recruiting agencies mirrored that. But by all our measures—customer account growth, longevity, profitability—the ones with strong BAs in the lead were clear winners.
Tristan was spot on too—the good ones are far, far better than the average. And maybe that is a factor in this. I can claim to be a capable “soft skill” person, and that is harder to disprove than some baseline of tech skills
Agreed, and there's not really even a model for paying people who are valuable well beyond the normal band. Sales teams do it with quotas and bonuses, but it's pretty rare for non-GTM positions to have that. I guess you get there with discretionary bonuses, though that feels pretty tricky to administer. (Facebook does this though, and I've heard of cases when some employees have gotten 100%+ of their salary in a bonus after a really big year.)
Here in Australia there is a divide. Engineers are likely to have a definite qualification, data scientists probably do (but do not assume too much) but for someone called an 'analyst' there is a culture in organisations that they do not need a qualification and just need to know some SQL. So we have a lot of people doing analytics who might find statistical thinking a challenge, and this overall detracts from the perceived value of the work of analysts. Engineers, though, command a price because they presumable could be doing something else of value.
Interesting - I've never thought about it from that angle. There's a lot of commentary about how analysts are hard to measure and their skills are hard to assess. This feels like a corollary to that that I've never considered, which is that analysts are also less consistent in what they do. Since the general perception of an analyst (which probably drives their pay) is a blend of of a lot of skillsets, the composite mash up of an analyst is pretty vague and potentially uninspiring.
Insightful take, Benn. Thank you.