This is great. And, playing devil's advocate, arguably the rarefied syntax of many programming languages - and the idea that a perfect language should eliminate verbosity - is in some ways structurally inimical to the organic process of creative problem-solving required for good analysis, which is often messier, more compromised.
Great piece, Benn - thanks for writing it. I absolutely agree that technical skills are (sometimes) necessary but certainly not sufficient for an analyst to be successful. I would add a couple of things: In many orgs, analytics is done by people who don't even hold the formal title of "Analyst" (i.e. people in stakeholder teams) - for them, it is their subject-matter expertise which is what gives them the ability to ask (and answer) interesting questions.
Secondly, I think it is relevant that the modern data stack forces all analysts to have to learn SQL or Python. Ironically enough, older tech like OLAP cubes, well-executed, can enable analysts to drill into data without having to do a lot of heavy lifting. These kinds of heavily modeled analytic datasets should, in my opinion, be the primary output of Analytics Engineers. I know that not every analytic task can be solved by turning to a nicely presented cube, but companies should focus on providing as many of these well-formed datasets as possible to enable inquisitive analysts to answer questions, whether they hold the formal title or not.
I largely agree with this post + felt it was well written. One bit I am sort of waffling over though is to what degree an analyst can be non-technical. One interpretation of what you wrote (albeit perhaps an extreme one) is that we hire people to just "think" all day about how to do things and share their thoughts with engineers who could implement.
I don't think I believe that you can be a critical thinker without being able to get the right visibility into the problem yourself. The (perhaps unfortunate) reality is that is near impossible to do on pen + paper today; and not even excel for some problems.
I think we should absolutely build better tooling to lower the barrier to entry / improve accessibility, but I don't think we will ever get away from needing some technical skill here; we'll just reduce our dependence on it.
Thanks for sharing your thoughts, really enjoyed the read!
Thank you for writing this. I feel this article perfectly encapsulates the struggle I face as a recovering academic trying to break into the realm of data science (and a female one at that), and communicate the potential contribution I can make to a team. It's hard to get your foot in the door when the only metric you are judged by is the list of programming languages at the bottom of your resume. And the resume is being judged by individuals who don't even know how to pose the questions they need to answer. I want to shout "What is your question? Tell me and I'll figure out how to use whatever tool necessary to answer it!"
This is great, but how do we find these expert "non-technical" analytical candidates? What skills and experience are they going to have on their resume that HR will be able to spot? I would gladly hire someone with basic SQL and python/R knowledge if they have all the other soft skills in abundance, but how do we find those when searching through potential candidates at scale?
From someone who stands at this crossroads today, thank you for this article Benn! This was actually a topic I brought up with my manager recently: As someone senior in the company, what are my career options moving forward?
In my company at least, I already feel that the only concrete way forward is the "analytics engineer." When your communication skills are "good enough", it's very tempting to pick up technical skills instead. There are plenty of resources already to guide people in my position to go the way of the analytics engineer.
However, I really think I would thrive as an analytical specialist, as you put it. What areas or skills would you say differentiate a junior analyst from a more senior analytical specialist? I do harbor the same belief that analytics is tech-agnostic! I'm just lost on what I need to develop further to walk down the path of an analytical specialist.
Benn, your insanely eloquent article popped up on my smartphone this morning, sandwiched as a "read more" between awfully similarly titled towardsdatascience posts on how many mistakes I'll make in my job or career or life (mostly, I'm a data mistaker) if I don't use some very specific Python functions. I'm so glad I clicked on your article instead - what a treat!
You are spot on. I'm probably one of a few classical musicians turned data detectives of late and still questioning the sanity of the new field (the old is clearly insane). I'm coming to the meta realization that this questioning work... IS THE WORK? (hopefully the CAPS had a deeper resounding tone).
By the way, the second link below the column chart (322.50) leads to a fun copy-paste mistake URL - but please don't fix that since it was way more fun to discover how that URL came to be than any other URL you could've possibly come up with. Thanks!
I agree so strongly with everything in this piece. The tech industry badly needs more diversity in every way, including in its workers' educational backgrounds and professional experiences. As a newly minted data scientist, I found that my 20 years of domain experience in another industry was very undervalued in tech, even by companies that expressly serve the industry in which I had worked. It makes no sense! The companies that figure this out and hire people from outside of tech are going to have a huge advantage in the coming years.
You set up a straw man argument. I don't disagree with anything you said, but you didn't go out of your way to look it at from the company's side, nor did you address your own assumptions.
The number one rock star programmer I ever worked with had one semester of community college. I know well a rock star facilities engineer on a local college campus, who can't write. The best CEO to hire might be a high school dropout. We all know exceptions exist in every field. I don't see what's particularly special about passed over analysts.
Companies do want the best people, degrees or not. They just don't have the time to carefully look at every applicant. A selling point of a degree is winnowing the interview schedule to 10 people, not 238. Even if you were to wave a magic wand so every employer took your hiring practice suggestions, some good ones would be passed over.
Your assumption is anything can be learned. I hope I'm paraphrasing you correctly: "You know how to solve problems--real problems? Great! Don't worry about knowing python or SQL, you can learn that." MAYBE. MAYBE NOT. That's a big risk for the company to take on. I know two SQL wizards who couldn't learn LDAP in 10 years to save their job. I know 3 rock star assembly programmers on a mainframe, 2 of which couldn't transition to C# on PCs.
Again, I'll try to paraphrase you accurately: "You need a good analyst who can find the hidden causes and connections--not a punch list of tools and skills that might not succeed. Who cares if the analyst uses python or a spread sheet?" The assumption is that the lone spread sheet wizard can deliver ALL the answers. Imagine the spreadsheeter finds a gem. The company leaders are thrilled. They ask some great follow up questions the spreadsheeter didn't think about--there are only so many hours in the day. The spreadsheeter says, "Great questions. I'll get on those when I get back from vacation 3 weeks from now. I'll give my sheets to Frank and Maria. They can answer those." SERIOUSLY? Sometimes common skills and tools are not the best choice for a particular job, but being common, they allow sharing.
Good people being passed over is true and sad. The hit/miss ratio could be improved but is ultimately not solvable. That good people can learn anything they don't know is a bad, bad assumption. You won't understand that until you, yourself, hit a learning wall. The lone wolf isn't as helpful as a wolf pack.
quantitative vs qualitative : the technical skill set is easier to prove, and progress through, it is measured, and since most resume's go through a machine based on matched words and an easy to explain skill set, non-technical folks will be screened out by the computer long before human eyes are on the resume. Data Science has value because it is finding something new and as yet undefined in the data - so how do you put a skill set down for that in your requirements or on the program screening resumes? Pre-screened resumes by program can not find new and as yet undefined skill sets for jobs.
This is great - I totally agree. Most of the value I see analysts create is because they know a huge amount about their topic, not because they're great at slicing and dicing dataframes.
I'm not sure how well that works at most companies though, because there's much more demand for fast answers to the common questions, rather than great answers to something hard.
This is great. And, playing devil's advocate, arguably the rarefied syntax of many programming languages - and the idea that a perfect language should eliminate verbosity - is in some ways structurally inimical to the organic process of creative problem-solving required for good analysis, which is often messier, more compromised.
Great piece, Benn - thanks for writing it. I absolutely agree that technical skills are (sometimes) necessary but certainly not sufficient for an analyst to be successful. I would add a couple of things: In many orgs, analytics is done by people who don't even hold the formal title of "Analyst" (i.e. people in stakeholder teams) - for them, it is their subject-matter expertise which is what gives them the ability to ask (and answer) interesting questions.
Secondly, I think it is relevant that the modern data stack forces all analysts to have to learn SQL or Python. Ironically enough, older tech like OLAP cubes, well-executed, can enable analysts to drill into data without having to do a lot of heavy lifting. These kinds of heavily modeled analytic datasets should, in my opinion, be the primary output of Analytics Engineers. I know that not every analytic task can be solved by turning to a nicely presented cube, but companies should focus on providing as many of these well-formed datasets as possible to enable inquisitive analysts to answer questions, whether they hold the formal title or not.
I largely agree with this post + felt it was well written. One bit I am sort of waffling over though is to what degree an analyst can be non-technical. One interpretation of what you wrote (albeit perhaps an extreme one) is that we hire people to just "think" all day about how to do things and share their thoughts with engineers who could implement.
I don't think I believe that you can be a critical thinker without being able to get the right visibility into the problem yourself. The (perhaps unfortunate) reality is that is near impossible to do on pen + paper today; and not even excel for some problems.
I think we should absolutely build better tooling to lower the barrier to entry / improve accessibility, but I don't think we will ever get away from needing some technical skill here; we'll just reduce our dependence on it.
Thanks for sharing your thoughts, really enjoyed the read!
Thank you for writing this. I feel this article perfectly encapsulates the struggle I face as a recovering academic trying to break into the realm of data science (and a female one at that), and communicate the potential contribution I can make to a team. It's hard to get your foot in the door when the only metric you are judged by is the list of programming languages at the bottom of your resume. And the resume is being judged by individuals who don't even know how to pose the questions they need to answer. I want to shout "What is your question? Tell me and I'll figure out how to use whatever tool necessary to answer it!"
This is great, but how do we find these expert "non-technical" analytical candidates? What skills and experience are they going to have on their resume that HR will be able to spot? I would gladly hire someone with basic SQL and python/R knowledge if they have all the other soft skills in abundance, but how do we find those when searching through potential candidates at scale?
From someone who stands at this crossroads today, thank you for this article Benn! This was actually a topic I brought up with my manager recently: As someone senior in the company, what are my career options moving forward?
In my company at least, I already feel that the only concrete way forward is the "analytics engineer." When your communication skills are "good enough", it's very tempting to pick up technical skills instead. There are plenty of resources already to guide people in my position to go the way of the analytics engineer.
However, I really think I would thrive as an analytical specialist, as you put it. What areas or skills would you say differentiate a junior analyst from a more senior analytical specialist? I do harbor the same belief that analytics is tech-agnostic! I'm just lost on what I need to develop further to walk down the path of an analytical specialist.
I cannot agree more with this, thanks Benn! There's also a lot of chatter about what it means to be a "Data Person" -- I recently wrote about it and would love to know what you think: https://news.dataled.academy/issues/data-person-who-dat-670055
Benn, your insanely eloquent article popped up on my smartphone this morning, sandwiched as a "read more" between awfully similarly titled towardsdatascience posts on how many mistakes I'll make in my job or career or life (mostly, I'm a data mistaker) if I don't use some very specific Python functions. I'm so glad I clicked on your article instead - what a treat!
You are spot on. I'm probably one of a few classical musicians turned data detectives of late and still questioning the sanity of the new field (the old is clearly insane). I'm coming to the meta realization that this questioning work... IS THE WORK? (hopefully the CAPS had a deeper resounding tone).
By the way, the second link below the column chart (322.50) leads to a fun copy-paste mistake URL - but please don't fix that since it was way more fun to discover how that URL came to be than any other URL you could've possibly come up with. Thanks!
I agree so strongly with everything in this piece. The tech industry badly needs more diversity in every way, including in its workers' educational backgrounds and professional experiences. As a newly minted data scientist, I found that my 20 years of domain experience in another industry was very undervalued in tech, even by companies that expressly serve the industry in which I had worked. It makes no sense! The companies that figure this out and hire people from outside of tech are going to have a huge advantage in the coming years.
You set up a straw man argument. I don't disagree with anything you said, but you didn't go out of your way to look it at from the company's side, nor did you address your own assumptions.
The number one rock star programmer I ever worked with had one semester of community college. I know well a rock star facilities engineer on a local college campus, who can't write. The best CEO to hire might be a high school dropout. We all know exceptions exist in every field. I don't see what's particularly special about passed over analysts.
Companies do want the best people, degrees or not. They just don't have the time to carefully look at every applicant. A selling point of a degree is winnowing the interview schedule to 10 people, not 238. Even if you were to wave a magic wand so every employer took your hiring practice suggestions, some good ones would be passed over.
Your assumption is anything can be learned. I hope I'm paraphrasing you correctly: "You know how to solve problems--real problems? Great! Don't worry about knowing python or SQL, you can learn that." MAYBE. MAYBE NOT. That's a big risk for the company to take on. I know two SQL wizards who couldn't learn LDAP in 10 years to save their job. I know 3 rock star assembly programmers on a mainframe, 2 of which couldn't transition to C# on PCs.
Again, I'll try to paraphrase you accurately: "You need a good analyst who can find the hidden causes and connections--not a punch list of tools and skills that might not succeed. Who cares if the analyst uses python or a spread sheet?" The assumption is that the lone spread sheet wizard can deliver ALL the answers. Imagine the spreadsheeter finds a gem. The company leaders are thrilled. They ask some great follow up questions the spreadsheeter didn't think about--there are only so many hours in the day. The spreadsheeter says, "Great questions. I'll get on those when I get back from vacation 3 weeks from now. I'll give my sheets to Frank and Maria. They can answer those." SERIOUSLY? Sometimes common skills and tools are not the best choice for a particular job, but being common, they allow sharing.
Good people being passed over is true and sad. The hit/miss ratio could be improved but is ultimately not solvable. That good people can learn anything they don't know is a bad, bad assumption. You won't understand that until you, yourself, hit a learning wall. The lone wolf isn't as helpful as a wolf pack.
quantitative vs qualitative : the technical skill set is easier to prove, and progress through, it is measured, and since most resume's go through a machine based on matched words and an easy to explain skill set, non-technical folks will be screened out by the computer long before human eyes are on the resume. Data Science has value because it is finding something new and as yet undefined in the data - so how do you put a skill set down for that in your requirements or on the program screening resumes? Pre-screened resumes by program can not find new and as yet undefined skill sets for jobs.
This is great - I totally agree. Most of the value I see analysts create is because they know a huge amount about their topic, not because they're great at slicing and dicing dataframes.
I'm not sure how well that works at most companies though, because there's much more demand for fast answers to the common questions, rather than great answers to something hard.