This is great. And, playing devil's advocate, arguably the rarefied syntax of many programming languages - and the idea that a perfect language should eliminate verbosity - is in some ways structurally inimical to the organic process of creative problem-solving required for good analysis, which is often messier, more compromised.
This is a good point - for analysts at least, the point of SQL is to navigate a messy, meandering process (wrote about that a bit here: https://benn.substack.com/p/analytics-is-a-mess). Though it's great that process creates succinct code, that's not really the point.
Great piece, Benn - thanks for writing it. I absolutely agree that technical skills are (sometimes) necessary but certainly not sufficient for an analyst to be successful. I would add a couple of things: In many orgs, analytics is done by people who don't even hold the formal title of "Analyst" (i.e. people in stakeholder teams) - for them, it is their subject-matter expertise which is what gives them the ability to ask (and answer) interesting questions.
Secondly, I think it is relevant that the modern data stack forces all analysts to have to learn SQL or Python. Ironically enough, older tech like OLAP cubes, well-executed, can enable analysts to drill into data without having to do a lot of heavy lifting. These kinds of heavily modeled analytic datasets should, in my opinion, be the primary output of Analytics Engineers. I know that not every analytic task can be solved by turning to a nicely presented cube, but companies should focus on providing as many of these well-formed datasets as possible to enable inquisitive analysts to answer questions, whether they hold the formal title or not.
Thanks Ian! And I agree with that prescription, at least in form. I'm very on board with data teams providing paths for domain experts to ask their own questions, be curious, etc. Those are the folks who understand the problems, and can actually *be* curious. That said, I'm not sure that cubes or datasets are necessarily the right means for that. Other interfaces (eg, something oriented around metrics rather than tables) might actually work better for some people or problems. But that's more of a tactical nitpick.
I largely agree with this post + felt it was well written. One bit I am sort of waffling over though is to what degree an analyst can be non-technical. One interpretation of what you wrote (albeit perhaps an extreme one) is that we hire people to just "think" all day about how to do things and share their thoughts with engineers who could implement.
I don't think I believe that you can be a critical thinker without being able to get the right visibility into the problem yourself. The (perhaps unfortunate) reality is that is near impossible to do on pen + paper today; and not even excel for some problems.
I think we should absolutely build better tooling to lower the barrier to entry / improve accessibility, but I don't think we will ever get away from needing some technical skill here; we'll just reduce our dependence on it.
Thanks for sharing your thoughts, really enjoyed the read!
Thanks! And I don’t disagree that it’s impractical (if not impossible) to be an analyst today without knowing a bit of technical skill. But people can learn it on the job, so I don’t think we need to gate hiring on it.
For example, it’s also impossible to do your job without knowing your coworkers names. We obviously don’t require candidates to know that before they join; they’ll learn it once they start. IJust because you need to know it doesn’t mean you need to know it the day you start.
Agree. I wonder how you can help bridge that gap. While they certainly can learn it on the job, I wonder what the ramp up looks like (not saying it's long, I truly have no idea what it looks like). From the perspective of an employer who maybe spends months trying to hire a good person they needed yesterday, I can imagine the idea of a longer time to ramp up is undesirable.
Though, also worth noting that if we didn't gate on the technical skills to begin with, we would have a larger pool + easier time of hiring and could possibly shift the fees we pay for recruiting to training/upskilling our new analysts.
I think that that last point is a good one, especially given how much people struggle to hire for data roles. There’s an almost reflexive assumption that hiring data people is always going to be really hard, and I’m not sure it has to be that way.
Thank you for writing this. I feel this article perfectly encapsulates the struggle I face as a recovering academic trying to break into the realm of data science (and a female one at that), and communicate the potential contribution I can make to a team. It's hard to get your foot in the door when the only metric you are judged by is the list of programming languages at the bottom of your resume. And the resume is being judged by individuals who don't even know how to pose the questions they need to answer. I want to shout "What is your question? Tell me and I'll figure out how to use whatever tool necessary to answer it!"
This is great, but how do we find these expert "non-technical" analytical candidates? What skills and experience are they going to have on their resume that HR will be able to spot? I would gladly hire someone with basic SQL and python/R knowledge if they have all the other soft skills in abundance, but how do we find those when searching through potential candidates at scale?
I have the same question, but backwards. How can I express my soft skills or analytical mindset on my resume in a way that gives me the opportunity to have an interview where I demonstrate them?
On your question Joe, I don't think there's a great way to do this today. The best signals I typically see (which still aren't great) are the types of experiences people have, both professionally and personally. If people have spent time solving business problems, particularly in organizations that likely didn't have a ton of structure, I think that's usually a sign that they're flexible and curious. That's also true for academic backgrounds - open-ended subjects that require people to reason and write are often better signals than ones focused on hard math.
For me, the best signal is personal work though. If people have blogs or have tried to analyze public data they're interested in, you can get a really good sense of how they approach problems. It also shows that they're interested in that sort of work and aren't chasing data roles for how they're advertised, which is a huge plus.
On your question Nathaniel, I'd say something similar. If you have experiences solving problems, focus on those problems - who you worked with, what questions you answered, how you worked through those questions - rather than the packages you know or things like that. (Also, personally, for roles that aren't senior leaders, I think notes on resumes like "increased revenue by x%" aren't helpful. It's much more useful to know how you work on problems than how you can sell the result.)
Finally, for the same reasons I mentioned earlier, if you have a blog or personal projects you can point to, those are great.
Thanks to this post, I realized my open req is everything wrong with job reqs for analytics. I went back to HR this AM and got approval to try diversifying the req. Do you have any techniques you'd be willing to share as to how to craft job descriptions so as to attract these non-traditional-but-fully-qualified candidates?
Thanks - glad it was useful! In general, there are three things I'd try to do with job reqs:
1.) Don't be prescriptive about prior experience, particularly years of experience. Describe what you want people to do, and let folks decide if they're able to do it. Things like "3+ years experience working as operations analyst..." will discourage folks who could be really good from applying because they don't meet the exact qualifications, like they have only 1 year, or they have a few years working in an adjacent role but no that one (and, underrepresented candidates tend to more likely to not apply because of this).
2.) For the same reasons, don't put specific language or technical requirements. Again, you can say what people will be doing (eg, "You'll be working with data in Redshift to figure stuff out,"). But if you say, "must be proficient in SQL," a lot of people will weed themselves out, and may do so because their definition of proficient isn't the same as yours. (You can also be explicit about NOT needing to know these things, which can help as well. Something like "SQL is a big part of the job, but if you're not comfortable with SQL, we can help train you." Obviously, don't say that if you can't train them, but I've found a lot of companies expect to do on-the-job training, but don't tell anyone that in their reqs.
3.) Don't put nice-to-haves and things like that. A lot of people take nice-to-haves as must-haves. If you prefer people with Python experience, you can still hire the people with Python experience. But if it's not absolutely required, then don't put in on the req.
From someone who stands at this crossroads today, thank you for this article Benn! This was actually a topic I brought up with my manager recently: As someone senior in the company, what are my career options moving forward?
In my company at least, I already feel that the only concrete way forward is the "analytics engineer." When your communication skills are "good enough", it's very tempting to pick up technical skills instead. There are plenty of resources already to guide people in my position to go the way of the analytics engineer.
However, I really think I would thrive as an analytical specialist, as you put it. What areas or skills would you say differentiate a junior analyst from a more senior analytical specialist? I do harbor the same belief that analytics is tech-agnostic! I'm just lost on what I need to develop further to walk down the path of an analytical specialist.
That's tough. Working in the analytics engineer direction likely has more concrete next steps (learn this tool, that language), and those skills are often more clearly understood by people as something worthy of a bigger title, a raise, etc. Without things like that to point to, it can sometimes be tough to show people you're becoming more senior as an analyst. (Part of that depends on the company, and how familiar they are with what you do. If people see analytics as being responsible for building dashboards, it's a lot harder to get recognized than it is at a company that wants data roles to be more strategic.)
So what can you do, even at the dashboard heavy companies? I think there are a couple things.
First, pay attention to the things that people want to know, and try to answer those questions. If you see that people are wondering things aloud in meetings, and those things would help them make real decisions (as opposed to being something they're just curious about) make a note of it and see if you can answer it. If you're able to help people understand things that want to know and wouldn't know otherwise, people will want you in more and more important conversations.
Second, learn about the business and what people around the business need. The more you understand their world, the more you can proactively help, and connect their problems with the data that you understand. Senior roles in analytics start to look a lot like strategy roles, where you're involved in making big decisions for the company. And to do that, you need to understand how the business works, as well as how data works.
So, to answer your question more directly, I think the skills that differentiate junior and senior analysts are often those that help them be better partners with the business. It's knowing the business problems, being able to work with other people really well, and do all the things that make what you know about data useful to other leaders around the company.
On the data person question, I think of it like writing. Everyone has to write, so you could claim that every job is a writing job. But if someone says they have a writing job, I assume it's the main function of what they do - they're a journalist or a copywriter or a content manager on a marketing team or something. In other words, writing is a part of every job, but not every job is a writing job. I'd say the same thing for data.
Thanks Benn! So you'd agree that someone who uses data in their day to day can call themselves a data person even though it's not their job to collect, ingest, model the data, is that right?
Yeah, I don't think it's wrong, though I think it's a little misleading. I can call myself a writer because I occasionally write rants on a blog on the internet, and I'm not lying. But if I go to a party and tell someone I'm a writer, they'll probably think I mean something else.
I know what you mean. However, since "data person" is not a role, it's safe to assume that nobody will say they are a data person when asked what they do. My goal is to enable "less-technical" folks who work with data to feel comfortable asking questions about the data made available to them as well as feel confident identifying as a data person even though they might not be an engineer.
Ah, I like that. It's less about people identifying their roles as a Data Role, and more about people being comfortable saying, yeah, I know how to work with data, and am not intimidated when people ask me to use it.
Benn, your insanely eloquent article popped up on my smartphone this morning, sandwiched as a "read more" between awfully similarly titled towardsdatascience posts on how many mistakes I'll make in my job or career or life (mostly, I'm a data mistaker) if I don't use some very specific Python functions. I'm so glad I clicked on your article instead - what a treat!
You are spot on. I'm probably one of a few classical musicians turned data detectives of late and still questioning the sanity of the new field (the old is clearly insane). I'm coming to the meta realization that this questioning work... IS THE WORK? (hopefully the CAPS had a deeper resounding tone).
By the way, the second link below the column chart (322.50) leads to a fun copy-paste mistake URL - but please don't fix that since it was way more fun to discover how that URL came to be than any other URL you could've possibly come up with. Thanks!
Thanks! And don't worry, this is all part of the long game: Write some articles, build an audience, organize a meetup, meet people in person and over the course of a few meetups, become friends, invite everyone to a group dinner, and then, over dinner...tell them why they have to use my new python library.
I agree so strongly with everything in this piece. The tech industry badly needs more diversity in every way, including in its workers' educational backgrounds and professional experiences. As a newly minted data scientist, I found that my 20 years of domain experience in another industry was very undervalued in tech, even by companies that expressly serve the industry in which I had worked. It makes no sense! The companies that figure this out and hire people from outside of tech are going to have a huge advantage in the coming years.
You set up a straw man argument. I don't disagree with anything you said, but you didn't go out of your way to look it at from the company's side, nor did you address your own assumptions.
The number one rock star programmer I ever worked with had one semester of community college. I know well a rock star facilities engineer on a local college campus, who can't write. The best CEO to hire might be a high school dropout. We all know exceptions exist in every field. I don't see what's particularly special about passed over analysts.
Companies do want the best people, degrees or not. They just don't have the time to carefully look at every applicant. A selling point of a degree is winnowing the interview schedule to 10 people, not 238. Even if you were to wave a magic wand so every employer took your hiring practice suggestions, some good ones would be passed over.
Your assumption is anything can be learned. I hope I'm paraphrasing you correctly: "You know how to solve problems--real problems? Great! Don't worry about knowing python or SQL, you can learn that." MAYBE. MAYBE NOT. That's a big risk for the company to take on. I know two SQL wizards who couldn't learn LDAP in 10 years to save their job. I know 3 rock star assembly programmers on a mainframe, 2 of which couldn't transition to C# on PCs.
Again, I'll try to paraphrase you accurately: "You need a good analyst who can find the hidden causes and connections--not a punch list of tools and skills that might not succeed. Who cares if the analyst uses python or a spread sheet?" The assumption is that the lone spread sheet wizard can deliver ALL the answers. Imagine the spreadsheeter finds a gem. The company leaders are thrilled. They ask some great follow up questions the spreadsheeter didn't think about--there are only so many hours in the day. The spreadsheeter says, "Great questions. I'll get on those when I get back from vacation 3 weeks from now. I'll give my sheets to Frank and Maria. They can answer those." SERIOUSLY? Sometimes common skills and tools are not the best choice for a particular job, but being common, they allow sharing.
Good people being passed over is true and sad. The hit/miss ratio could be improved but is ultimately not solvable. That good people can learn anything they don't know is a bad, bad assumption. You won't understand that until you, yourself, hit a learning wall. The lone wolf isn't as helpful as a wolf pack.
I don’t expect hiring to be perfect, or good people to be able to learn anything. My argument is simply that, if we bucket the skills required to work in analytics into two categories - A) tech skills and B) whatever we call the problem solving one - we tend to market, hire for, talk about, and generally celebrate A. I believe that’s the less important skill of the two, in part because you can be a great analyst without A and not without B, and in part because A is generally easier to learn. That doesn’t mean someone who’s good at B can learn anything or that being good at A doesn’t help you. It just means that we filter out a lot of good people by screening aggressively for A.
I’m sure this happens in other fields too. Data is just the one I know about.
You drove straight to the defining issue with "and B) whatever we call the problem solving one". It's damn hard to hire for a quality without a name. Even harder to measure a quality without a name. Harder still is coming up with a simple meaningful name. Zen and the Art of Motorcycle Maintenance is an EXCELLENT book devoted to the subject.
Hiring is hard for both the candidate and the company. As a candidate, you only have control over you. It's easy to complain about the company's mistakes. What you said resonates with folks because it's true. I felt it.
So what? You can lament the process, shake a fist at the employment gods, and write a well-written blog. Describing the problem accurately only gets you so far, specifically denial, anger, bargaining, and depression. A great analyst moves on to acceptance, to working out a solution that overcomes the problem.
We can choose to be naïve like the brilliant Nikola Tesla who hated communicating in business and died resentful and penniless. Or we can choose to work within the idiosyncrasies of business, be successful, and maybe die rich like Thomas Edison (a real bastard--let's ignore that for now). You got passed over? Quit stewing. Fix it by fixing YOUR part in the communication.
No question the company's hiring practice sucks. Don't blame them for the communication gap even when they made the gap--especially if they made the gap. The blame absolutely stops your progress. Acceptance means understanding YOU have to jump the gap, not them. (Don't confuse accepting your responsibility to overcome nonsense obstacles with accepting the nonsense itself. Acceptance is not calling bad good.)
Look at your replies. Some still ask for the company to change like altering the language in the job posting--blame. Some discuss what you can do like building an analyst portfolio--acceptance. The blame game is cathartic and popular, but immature. I wouldn't hire you--yet.
quantitative vs qualitative : the technical skill set is easier to prove, and progress through, it is measured, and since most resume's go through a machine based on matched words and an easy to explain skill set, non-technical folks will be screened out by the computer long before human eyes are on the resume. Data Science has value because it is finding something new and as yet undefined in the data - so how do you put a skill set down for that in your requirements or on the program screening resumes? Pre-screened resumes by program can not find new and as yet undefined skill sets for jobs.
Yeah, agree that is really tough, though I think we could solve it if we wanted to. One idea that I'm warming to is the idea of an analyst portfolio, like a design portfolio: https://twitter.com/bennstancil/status/1421200319678136320
This is great - I totally agree. Most of the value I see analysts create is because they know a huge amount about their topic, not because they're great at slicing and dicing dataframes.
I'm not sure how well that works at most companies though, because there's much more demand for fast answers to the common questions, rather than great answers to something hard.
Thanks! And that makes sense, though if that's right, it feels like we need to do more to correct that problem. If answering those questions isn't that valuable (and for the most part, I don't think it is), we'd need to either just tell people we're not going to do it, or show people that there's something we can do that's way more valuable.
I don't have any easy answers for how to do this, and can't help but wonder if the real problem is what I was talking about in last week's post: Doing more providing fast answers to common questions is really hard, and probably harder than we give credit for. More often than not, when you ask people why they don't answer big questions, it's about time, or the organization not asking for it, or whatever. True as that may be, I'm not sure that our frequent inability to actually answer big questions isn't a bigger reason why we don't do it.
This is great. And, playing devil's advocate, arguably the rarefied syntax of many programming languages - and the idea that a perfect language should eliminate verbosity - is in some ways structurally inimical to the organic process of creative problem-solving required for good analysis, which is often messier, more compromised.
This is a good point - for analysts at least, the point of SQL is to navigate a messy, meandering process (wrote about that a bit here: https://benn.substack.com/p/analytics-is-a-mess). Though it's great that process creates succinct code, that's not really the point.
Great piece, Benn - thanks for writing it. I absolutely agree that technical skills are (sometimes) necessary but certainly not sufficient for an analyst to be successful. I would add a couple of things: In many orgs, analytics is done by people who don't even hold the formal title of "Analyst" (i.e. people in stakeholder teams) - for them, it is their subject-matter expertise which is what gives them the ability to ask (and answer) interesting questions.
Secondly, I think it is relevant that the modern data stack forces all analysts to have to learn SQL or Python. Ironically enough, older tech like OLAP cubes, well-executed, can enable analysts to drill into data without having to do a lot of heavy lifting. These kinds of heavily modeled analytic datasets should, in my opinion, be the primary output of Analytics Engineers. I know that not every analytic task can be solved by turning to a nicely presented cube, but companies should focus on providing as many of these well-formed datasets as possible to enable inquisitive analysts to answer questions, whether they hold the formal title or not.
Thanks Ian! And I agree with that prescription, at least in form. I'm very on board with data teams providing paths for domain experts to ask their own questions, be curious, etc. Those are the folks who understand the problems, and can actually *be* curious. That said, I'm not sure that cubes or datasets are necessarily the right means for that. Other interfaces (eg, something oriented around metrics rather than tables) might actually work better for some people or problems. But that's more of a tactical nitpick.
I largely agree with this post + felt it was well written. One bit I am sort of waffling over though is to what degree an analyst can be non-technical. One interpretation of what you wrote (albeit perhaps an extreme one) is that we hire people to just "think" all day about how to do things and share their thoughts with engineers who could implement.
I don't think I believe that you can be a critical thinker without being able to get the right visibility into the problem yourself. The (perhaps unfortunate) reality is that is near impossible to do on pen + paper today; and not even excel for some problems.
I think we should absolutely build better tooling to lower the barrier to entry / improve accessibility, but I don't think we will ever get away from needing some technical skill here; we'll just reduce our dependence on it.
Thanks for sharing your thoughts, really enjoyed the read!
Thanks! And I don’t disagree that it’s impractical (if not impossible) to be an analyst today without knowing a bit of technical skill. But people can learn it on the job, so I don’t think we need to gate hiring on it.
For example, it’s also impossible to do your job without knowing your coworkers names. We obviously don’t require candidates to know that before they join; they’ll learn it once they start. IJust because you need to know it doesn’t mean you need to know it the day you start.
Agree. I wonder how you can help bridge that gap. While they certainly can learn it on the job, I wonder what the ramp up looks like (not saying it's long, I truly have no idea what it looks like). From the perspective of an employer who maybe spends months trying to hire a good person they needed yesterday, I can imagine the idea of a longer time to ramp up is undesirable.
Though, also worth noting that if we didn't gate on the technical skills to begin with, we would have a larger pool + easier time of hiring and could possibly shift the fees we pay for recruiting to training/upskilling our new analysts.
I think that that last point is a good one, especially given how much people struggle to hire for data roles. There’s an almost reflexive assumption that hiring data people is always going to be really hard, and I’m not sure it has to be that way.
Thank you for writing this. I feel this article perfectly encapsulates the struggle I face as a recovering academic trying to break into the realm of data science (and a female one at that), and communicate the potential contribution I can make to a team. It's hard to get your foot in the door when the only metric you are judged by is the list of programming languages at the bottom of your resume. And the resume is being judged by individuals who don't even know how to pose the questions they need to answer. I want to shout "What is your question? Tell me and I'll figure out how to use whatever tool necessary to answer it!"
This is great, but how do we find these expert "non-technical" analytical candidates? What skills and experience are they going to have on their resume that HR will be able to spot? I would gladly hire someone with basic SQL and python/R knowledge if they have all the other soft skills in abundance, but how do we find those when searching through potential candidates at scale?
I have the same question, but backwards. How can I express my soft skills or analytical mindset on my resume in a way that gives me the opportunity to have an interview where I demonstrate them?
On your question Joe, I don't think there's a great way to do this today. The best signals I typically see (which still aren't great) are the types of experiences people have, both professionally and personally. If people have spent time solving business problems, particularly in organizations that likely didn't have a ton of structure, I think that's usually a sign that they're flexible and curious. That's also true for academic backgrounds - open-ended subjects that require people to reason and write are often better signals than ones focused on hard math.
For me, the best signal is personal work though. If people have blogs or have tried to analyze public data they're interested in, you can get a really good sense of how they approach problems. It also shows that they're interested in that sort of work and aren't chasing data roles for how they're advertised, which is a huge plus.
On your question Nathaniel, I'd say something similar. If you have experiences solving problems, focus on those problems - who you worked with, what questions you answered, how you worked through those questions - rather than the packages you know or things like that. (Also, personally, for roles that aren't senior leaders, I think notes on resumes like "increased revenue by x%" aren't helpful. It's much more useful to know how you work on problems than how you can sell the result.)
Finally, for the same reasons I mentioned earlier, if you have a blog or personal projects you can point to, those are great.
Thanks to this post, I realized my open req is everything wrong with job reqs for analytics. I went back to HR this AM and got approval to try diversifying the req. Do you have any techniques you'd be willing to share as to how to craft job descriptions so as to attract these non-traditional-but-fully-qualified candidates?
Thanks - glad it was useful! In general, there are three things I'd try to do with job reqs:
1.) Don't be prescriptive about prior experience, particularly years of experience. Describe what you want people to do, and let folks decide if they're able to do it. Things like "3+ years experience working as operations analyst..." will discourage folks who could be really good from applying because they don't meet the exact qualifications, like they have only 1 year, or they have a few years working in an adjacent role but no that one (and, underrepresented candidates tend to more likely to not apply because of this).
2.) For the same reasons, don't put specific language or technical requirements. Again, you can say what people will be doing (eg, "You'll be working with data in Redshift to figure stuff out,"). But if you say, "must be proficient in SQL," a lot of people will weed themselves out, and may do so because their definition of proficient isn't the same as yours. (You can also be explicit about NOT needing to know these things, which can help as well. Something like "SQL is a big part of the job, but if you're not comfortable with SQL, we can help train you." Obviously, don't say that if you can't train them, but I've found a lot of companies expect to do on-the-job training, but don't tell anyone that in their reqs.
3.) Don't put nice-to-haves and things like that. A lot of people take nice-to-haves as must-haves. If you prefer people with Python experience, you can still hire the people with Python experience. But if it's not absolutely required, then don't put in on the req.
From someone who stands at this crossroads today, thank you for this article Benn! This was actually a topic I brought up with my manager recently: As someone senior in the company, what are my career options moving forward?
In my company at least, I already feel that the only concrete way forward is the "analytics engineer." When your communication skills are "good enough", it's very tempting to pick up technical skills instead. There are plenty of resources already to guide people in my position to go the way of the analytics engineer.
However, I really think I would thrive as an analytical specialist, as you put it. What areas or skills would you say differentiate a junior analyst from a more senior analytical specialist? I do harbor the same belief that analytics is tech-agnostic! I'm just lost on what I need to develop further to walk down the path of an analytical specialist.
That's tough. Working in the analytics engineer direction likely has more concrete next steps (learn this tool, that language), and those skills are often more clearly understood by people as something worthy of a bigger title, a raise, etc. Without things like that to point to, it can sometimes be tough to show people you're becoming more senior as an analyst. (Part of that depends on the company, and how familiar they are with what you do. If people see analytics as being responsible for building dashboards, it's a lot harder to get recognized than it is at a company that wants data roles to be more strategic.)
So what can you do, even at the dashboard heavy companies? I think there are a couple things.
First, pay attention to the things that people want to know, and try to answer those questions. If you see that people are wondering things aloud in meetings, and those things would help them make real decisions (as opposed to being something they're just curious about) make a note of it and see if you can answer it. If you're able to help people understand things that want to know and wouldn't know otherwise, people will want you in more and more important conversations.
Second, learn about the business and what people around the business need. The more you understand their world, the more you can proactively help, and connect their problems with the data that you understand. Senior roles in analytics start to look a lot like strategy roles, where you're involved in making big decisions for the company. And to do that, you need to understand how the business works, as well as how data works.
So, to answer your question more directly, I think the skills that differentiate junior and senior analysts are often those that help them be better partners with the business. It's knowing the business problems, being able to work with other people really well, and do all the things that make what you know about data useful to other leaders around the company.
I cannot agree more with this, thanks Benn! There's also a lot of chatter about what it means to be a "Data Person" -- I recently wrote about it and would love to know what you think: https://news.dataled.academy/issues/data-person-who-dat-670055
Thanks!
On the data person question, I think of it like writing. Everyone has to write, so you could claim that every job is a writing job. But if someone says they have a writing job, I assume it's the main function of what they do - they're a journalist or a copywriter or a content manager on a marketing team or something. In other words, writing is a part of every job, but not every job is a writing job. I'd say the same thing for data.
Thanks Benn! So you'd agree that someone who uses data in their day to day can call themselves a data person even though it's not their job to collect, ingest, model the data, is that right?
Yeah, I don't think it's wrong, though I think it's a little misleading. I can call myself a writer because I occasionally write rants on a blog on the internet, and I'm not lying. But if I go to a party and tell someone I'm a writer, they'll probably think I mean something else.
I know what you mean. However, since "data person" is not a role, it's safe to assume that nobody will say they are a data person when asked what they do. My goal is to enable "less-technical" folks who work with data to feel comfortable asking questions about the data made available to them as well as feel confident identifying as a data person even though they might not be an engineer.
Ah, I like that. It's less about people identifying their roles as a Data Role, and more about people being comfortable saying, yeah, I know how to work with data, and am not intimidated when people ask me to use it.
Yeah exactly what you said!
Benn, your insanely eloquent article popped up on my smartphone this morning, sandwiched as a "read more" between awfully similarly titled towardsdatascience posts on how many mistakes I'll make in my job or career or life (mostly, I'm a data mistaker) if I don't use some very specific Python functions. I'm so glad I clicked on your article instead - what a treat!
You are spot on. I'm probably one of a few classical musicians turned data detectives of late and still questioning the sanity of the new field (the old is clearly insane). I'm coming to the meta realization that this questioning work... IS THE WORK? (hopefully the CAPS had a deeper resounding tone).
By the way, the second link below the column chart (322.50) leads to a fun copy-paste mistake URL - but please don't fix that since it was way more fun to discover how that URL came to be than any other URL you could've possibly come up with. Thanks!
Thanks! And don't worry, this is all part of the long game: Write some articles, build an audience, organize a meetup, meet people in person and over the course of a few meetups, become friends, invite everyone to a group dinner, and then, over dinner...tell them why they have to use my new python library.
I agree so strongly with everything in this piece. The tech industry badly needs more diversity in every way, including in its workers' educational backgrounds and professional experiences. As a newly minted data scientist, I found that my 20 years of domain experience in another industry was very undervalued in tech, even by companies that expressly serve the industry in which I had worked. It makes no sense! The companies that figure this out and hire people from outside of tech are going to have a huge advantage in the coming years.
Thanks! And I hope so. The shift to remote might help too, since that's pushing companies to think more broadly about how they hire.
You set up a straw man argument. I don't disagree with anything you said, but you didn't go out of your way to look it at from the company's side, nor did you address your own assumptions.
The number one rock star programmer I ever worked with had one semester of community college. I know well a rock star facilities engineer on a local college campus, who can't write. The best CEO to hire might be a high school dropout. We all know exceptions exist in every field. I don't see what's particularly special about passed over analysts.
Companies do want the best people, degrees or not. They just don't have the time to carefully look at every applicant. A selling point of a degree is winnowing the interview schedule to 10 people, not 238. Even if you were to wave a magic wand so every employer took your hiring practice suggestions, some good ones would be passed over.
Your assumption is anything can be learned. I hope I'm paraphrasing you correctly: "You know how to solve problems--real problems? Great! Don't worry about knowing python or SQL, you can learn that." MAYBE. MAYBE NOT. That's a big risk for the company to take on. I know two SQL wizards who couldn't learn LDAP in 10 years to save their job. I know 3 rock star assembly programmers on a mainframe, 2 of which couldn't transition to C# on PCs.
Again, I'll try to paraphrase you accurately: "You need a good analyst who can find the hidden causes and connections--not a punch list of tools and skills that might not succeed. Who cares if the analyst uses python or a spread sheet?" The assumption is that the lone spread sheet wizard can deliver ALL the answers. Imagine the spreadsheeter finds a gem. The company leaders are thrilled. They ask some great follow up questions the spreadsheeter didn't think about--there are only so many hours in the day. The spreadsheeter says, "Great questions. I'll get on those when I get back from vacation 3 weeks from now. I'll give my sheets to Frank and Maria. They can answer those." SERIOUSLY? Sometimes common skills and tools are not the best choice for a particular job, but being common, they allow sharing.
Good people being passed over is true and sad. The hit/miss ratio could be improved but is ultimately not solvable. That good people can learn anything they don't know is a bad, bad assumption. You won't understand that until you, yourself, hit a learning wall. The lone wolf isn't as helpful as a wolf pack.
I don’t expect hiring to be perfect, or good people to be able to learn anything. My argument is simply that, if we bucket the skills required to work in analytics into two categories - A) tech skills and B) whatever we call the problem solving one - we tend to market, hire for, talk about, and generally celebrate A. I believe that’s the less important skill of the two, in part because you can be a great analyst without A and not without B, and in part because A is generally easier to learn. That doesn’t mean someone who’s good at B can learn anything or that being good at A doesn’t help you. It just means that we filter out a lot of good people by screening aggressively for A.
I’m sure this happens in other fields too. Data is just the one I know about.
You drove straight to the defining issue with "and B) whatever we call the problem solving one". It's damn hard to hire for a quality without a name. Even harder to measure a quality without a name. Harder still is coming up with a simple meaningful name. Zen and the Art of Motorcycle Maintenance is an EXCELLENT book devoted to the subject.
Hiring is hard for both the candidate and the company. As a candidate, you only have control over you. It's easy to complain about the company's mistakes. What you said resonates with folks because it's true. I felt it.
So what? You can lament the process, shake a fist at the employment gods, and write a well-written blog. Describing the problem accurately only gets you so far, specifically denial, anger, bargaining, and depression. A great analyst moves on to acceptance, to working out a solution that overcomes the problem.
We can choose to be naïve like the brilliant Nikola Tesla who hated communicating in business and died resentful and penniless. Or we can choose to work within the idiosyncrasies of business, be successful, and maybe die rich like Thomas Edison (a real bastard--let's ignore that for now). You got passed over? Quit stewing. Fix it by fixing YOUR part in the communication.
No question the company's hiring practice sucks. Don't blame them for the communication gap even when they made the gap--especially if they made the gap. The blame absolutely stops your progress. Acceptance means understanding YOU have to jump the gap, not them. (Don't confuse accepting your responsibility to overcome nonsense obstacles with accepting the nonsense itself. Acceptance is not calling bad good.)
Look at your replies. Some still ask for the company to change like altering the language in the job posting--blame. Some discuss what you can do like building an analyst portfolio--acceptance. The blame game is cathartic and popular, but immature. I wouldn't hire you--yet.
quantitative vs qualitative : the technical skill set is easier to prove, and progress through, it is measured, and since most resume's go through a machine based on matched words and an easy to explain skill set, non-technical folks will be screened out by the computer long before human eyes are on the resume. Data Science has value because it is finding something new and as yet undefined in the data - so how do you put a skill set down for that in your requirements or on the program screening resumes? Pre-screened resumes by program can not find new and as yet undefined skill sets for jobs.
Yeah, agree that is really tough, though I think we could solve it if we wanted to. One idea that I'm warming to is the idea of an analyst portfolio, like a design portfolio: https://twitter.com/bennstancil/status/1421200319678136320
This is great - I totally agree. Most of the value I see analysts create is because they know a huge amount about their topic, not because they're great at slicing and dicing dataframes.
I'm not sure how well that works at most companies though, because there's much more demand for fast answers to the common questions, rather than great answers to something hard.
Thanks! And that makes sense, though if that's right, it feels like we need to do more to correct that problem. If answering those questions isn't that valuable (and for the most part, I don't think it is), we'd need to either just tell people we're not going to do it, or show people that there's something we can do that's way more valuable.
I don't have any easy answers for how to do this, and can't help but wonder if the real problem is what I was talking about in last week's post: Doing more providing fast answers to common questions is really hard, and probably harder than we give credit for. More often than not, when you ask people why they don't answer big questions, it's about time, or the organization not asking for it, or whatever. True as that may be, I'm not sure that our frequent inability to actually answer big questions isn't a bigger reason why we don't do it.