To me, this dynamic is the single most determinative fact on this topic. If I had some magical way to quantify the output of data analysts and analytics engineers, my gut instinct is that it would line up directly with your charts in that post. Which is to say: the median data analyst provides less value but the max data analyst provides dramatically _more value_ than an analytics engineer. The distributions look very different.
I know some of those top-end-of-the-distribution analysts and I could imagine easily making the argument that they should get paid $500k+ if they're working in a domain area that rewards it. And the thing that is special about those folks is, as you say, not their "technical" skills...it's their domain knowledge and "soft skills".
Sometimes traditional career-ladder-focused compensation practices don't do a good job of accurately valuing impact that is distributed in this way.
Preach it, brother! That’s why I’m pushing the term “data therapist” :-) The hardest work of all is helping people productively articulate what they truly want…
There's definitely a technical skill hierarchy in the data world's zeitgeist: SWE, Data > Data Engineer (platform) > Analytics Engineer > Data Analyst / Scientist / BI Developer. This is reflected both in salaries and community prestige. Analytics work is "easy" and "boring"; SQL development is "simple"; data engineers "use simplifying frameworks and tooling"; and SWEs, the real heroes, are writing complex distributed backends using cutting-edge algos and their enormous brains.
Of course this is nonsense; you can do some incredibly complex analyses and write some very simplistic event emission; then again, if you suggest that you can teach someone to write Kafka consumers more easily than you can teach someone to create a reliable forecast, a lot of people would bite your head off. If an analyst identifies $100M of revenue using a model an AE built that comes from data a DE landed and that the SWEs ultimately surfaced through the product, don't they all deserve credit for their respective parts in the value chain?
I end up interviewing lots of people who are very technical, a fair number of people who can write and communicate well, and very few people who embody both. I've found it's very hard to get HR to value that though.
I wonder if another dynamic is that the "analyst" title can span a broader range of capabilities than "analytics engineer"? The base level capability of an analytics engineer is already fairly high because there are certain things you need to be able to do to do your job. Whereas the base level of someone calling themselves an analyst is possibly lower, because the role is broader and more amorphous. So being able to make a pie chart in excel and have an opinion about what it means is probably enough to call yourself an analyst?
I don't at all mean to be disparaging. On the other end there are analysts absolutely doing very sophisticated work on the technical side and the analytical side and the business/product side and the people side. If anything, between analyst and analyst engineer, I'm an analyst.
I think there is maybe a similar but opposite problem with the data scientist title, because like analysts they are inter-disciplinary roles and need expertise in a pretty broad range of skills. Maybe the breadth of the role makes it too difficult to quantify?
So on the data scientist side (maybe because it started with PhDs? Don't know?) the breadth made people label it as fancy, whereas on the analyst side the breadth made people label it as unsophisticated / untechnical (also perhaps because it is a more female dominated field and that means it's easier, right?)
It takes constant learning and skills improvement to be a Systems/Data/Software Engineer. Platforms, frameworks, and the technology ecosystem are constantly changing. Most of the time, this learning effort is done outside of the regular work schedule at the sacrifice of a personal life because there just isn't enough time in a day to do it all. The amount of information and the toolset any one of these positions requires is overwhelming.
Jun 1, 2022·edited Jun 1, 2022Liked by Benn Stancil
We had a similar ongoing observation in my software development firm a few years ago. Our incoming business analysts (or product owners, or product managers, etc) generally expected less pay than software engineers. Market data provided by recruiting agencies mirrored that. But by all our measures—customer account growth, longevity, profitability—the ones with strong BAs in the lead were clear winners.
Tristan was spot on too—the good ones are far, far better than the average. And maybe that is a factor in this. I can claim to be a capable “soft skill” person, and that is harder to disprove than some baseline of tech skills
Here in Australia there is a divide. Engineers are likely to have a definite qualification, data scientists probably do (but do not assume too much) but for someone called an 'analyst' there is a culture in organisations that they do not need a qualification and just need to know some SQL. So we have a lot of people doing analytics who might find statistical thinking a challenge, and this overall detracts from the perceived value of the work of analysts. Engineers, though, command a price because they presumable could be doing something else of value.
Didn't you write a post about the work done by data analysts and compare it to honeydew? Ah yes, found it:
https://benn.substack.com/p/third-rail
To me, this dynamic is the single most determinative fact on this topic. If I had some magical way to quantify the output of data analysts and analytics engineers, my gut instinct is that it would line up directly with your charts in that post. Which is to say: the median data analyst provides less value but the max data analyst provides dramatically _more value_ than an analytics engineer. The distributions look very different.
I know some of those top-end-of-the-distribution analysts and I could imagine easily making the argument that they should get paid $500k+ if they're working in a domain area that rewards it. And the thing that is special about those folks is, as you say, not their "technical" skills...it's their domain knowledge and "soft skills".
Sometimes traditional career-ladder-focused compensation practices don't do a good job of accurately valuing impact that is distributed in this way.
Preach it, brother! That’s why I’m pushing the term “data therapist” :-) The hardest work of all is helping people productively articulate what they truly want…
As usual, the footnotes are (nearly) as good as the post itself. Thanks Benn.
Charity Majors has a great article that touches on your "more technical" point: https://charity.wtf/2022/01/20/how-engineering-driven-leads-to-engineering-supremacy/.
There's definitely a technical skill hierarchy in the data world's zeitgeist: SWE, Data > Data Engineer (platform) > Analytics Engineer > Data Analyst / Scientist / BI Developer. This is reflected both in salaries and community prestige. Analytics work is "easy" and "boring"; SQL development is "simple"; data engineers "use simplifying frameworks and tooling"; and SWEs, the real heroes, are writing complex distributed backends using cutting-edge algos and their enormous brains.
Of course this is nonsense; you can do some incredibly complex analyses and write some very simplistic event emission; then again, if you suggest that you can teach someone to write Kafka consumers more easily than you can teach someone to create a reliable forecast, a lot of people would bite your head off. If an analyst identifies $100M of revenue using a model an AE built that comes from data a DE landed and that the SWEs ultimately surfaced through the product, don't they all deserve credit for their respective parts in the value chain?
I end up interviewing lots of people who are very technical, a fair number of people who can write and communicate well, and very few people who embody both. I've found it's very hard to get HR to value that though.
I wonder if another dynamic is that the "analyst" title can span a broader range of capabilities than "analytics engineer"? The base level capability of an analytics engineer is already fairly high because there are certain things you need to be able to do to do your job. Whereas the base level of someone calling themselves an analyst is possibly lower, because the role is broader and more amorphous. So being able to make a pie chart in excel and have an opinion about what it means is probably enough to call yourself an analyst?
I don't at all mean to be disparaging. On the other end there are analysts absolutely doing very sophisticated work on the technical side and the analytical side and the business/product side and the people side. If anything, between analyst and analyst engineer, I'm an analyst.
I think there is maybe a similar but opposite problem with the data scientist title, because like analysts they are inter-disciplinary roles and need expertise in a pretty broad range of skills. Maybe the breadth of the role makes it too difficult to quantify?
So on the data scientist side (maybe because it started with PhDs? Don't know?) the breadth made people label it as fancy, whereas on the analyst side the breadth made people label it as unsophisticated / untechnical (also perhaps because it is a more female dominated field and that means it's easier, right?)
Just some musings!
It takes constant learning and skills improvement to be a Systems/Data/Software Engineer. Platforms, frameworks, and the technology ecosystem are constantly changing. Most of the time, this learning effort is done outside of the regular work schedule at the sacrifice of a personal life because there just isn't enough time in a day to do it all. The amount of information and the toolset any one of these positions requires is overwhelming.
We had a similar ongoing observation in my software development firm a few years ago. Our incoming business analysts (or product owners, or product managers, etc) generally expected less pay than software engineers. Market data provided by recruiting agencies mirrored that. But by all our measures—customer account growth, longevity, profitability—the ones with strong BAs in the lead were clear winners.
Tristan was spot on too—the good ones are far, far better than the average. And maybe that is a factor in this. I can claim to be a capable “soft skill” person, and that is harder to disprove than some baseline of tech skills
Here in Australia there is a divide. Engineers are likely to have a definite qualification, data scientists probably do (but do not assume too much) but for someone called an 'analyst' there is a culture in organisations that they do not need a qualification and just need to know some SQL. So we have a lot of people doing analytics who might find statistical thinking a challenge, and this overall detracts from the perceived value of the work of analysts. Engineers, though, command a price because they presumable could be doing something else of value.
Insightful take, Benn. Thank you.