Guest blog by Michael Young, CEO at MBN Solutions
MBN Solutions are our delivery partner for our MSc course, creating placement opportunities for MSc Data Lab Students. They will be at our DataTalent event in Glasgow on 17 March.
Despite the excitement of serving a growing and thriving Data Science sector in the UK, I’m hearing more disquiet. Hiring managers and candidates expressing doubts as to whether organisations really need another data scientist. Plus, experienced Data Leaders tell me how much they still value analysts even though they are “out of fashion”.
So, I was interested to see Luke Posey post an article on Towards Data Science that addresses this concern directly. In “Stop hiring Data Scientists”, Luke focusses especially on the missing Analysts, Statisticians & Data Engineers. The problem of too many organisations hiring another data scientist when that is not what they need.
Data Scientists like Luke are better qualified than me to confirm this misuse of the classic Data Science role. But I want to add my voice to the call for greater recognition of the breath of roles businesses should be considering.
At MBN we always seek to have an ongoing dialogue with those hiring, in part to better clarify such needs. I agree with Luke that the answer to avoiding hiring the wrong role is improved clarity. Being clear on both what your business (and customers) need from data & the potential roles that can help you.
Why this mislabelling matters
So, why does this matter? Surely businesses are free to hire who they want & use whichever job titles they prefer? Of course. In no way am I wanting to tell business leaders what to do. But there are some worrying signs that current hiring behaviour may be counterproductive.
Consider these statistics:
· Gartner estimates that 85% of data projects fail.
· Figure Eight reports that 55% of Data Scientists cite poor quality data as holding them back.
· New Vantage’s study shows that 85% of executives surveyed started a data-driven culture initiative, but only 37% have seen any success so far.
Clearly not all is well in the world of many businesses aspiring to be the next Google or Amazon. Despite ever increasing hiring of Data Scientists, too little progress is being made. In part, I believe this is due to the problems that Luke highlights in his post.
Hiring a Data Scientist when you really need an Analyst or Statistician is a recipe for frustration on both sides. Continuing to hire only Data Scientists (with demand outstripping supply) is driving higher and higher salaries for that role and greater frustration amongst others. As Luke highlights, this causes many analysts, statisticians or data engineers to change their label. It’s tempting to do so when higher salaries are on offer, but such mislabelling just adds to the confusion.
Without pretending to match the technical expertise of Data Scientists and other technicians, let me share some of what you may be missing. I will attempt to redress the balance by speaking up in praise of other roles. From our perspective, why might you want to hire a different role than a Data Scientist?
Reasons to hire an Analyst
The work of effective analysts is highly varied. They can turn their hand to SQL coding, diagnostic analysis, data visualisation, BI reporting & consultancy. Over the years I have been impressed to see skilled analysts solve many key problems in their businesses. I’ve also seen many more analysts rise to senior leadership roles than other technicians.
The benefits of hiring an analyst rather than a Data Scientist appear to be two-fold. Firstly, you get the flexibility and broad range of skills described above. Secondly, you avoid demotivating a Data Scientist, many of whom will see such tasks as too basic for them.
A few years ago, I saw a post from experienced Analytics leader Martin Squires that made this point well. In an interview he challenged the prejudice of seeing the alternative to a data scientist as “just an analyst”. Later he used the analogy of a medical General Practitioner (GP) to explain how useful they are compared to the few times you need to see a Surgeon.
If your key business need is improved reporting, marketing measurement or analysis to be presented to your Board – is it worth considering an Analyst rather than a Data Scientist?
Reasons to hire a Data Engineer
It is sad to read in surveys that Data Scientists & Analysts still spend most of their time working around data problems. Here at MBN we hear frustrations about missing data, poor quality data & just ‘really slow to get hold of’ data. Both managers and data scientists sound equally frustrated.
Surprisingly, sometimes the very organisations that express this frustration also plan to hire another data scientist. Thinking that will solve the issue. In more than one case I have pointed leaders to the post we published on why you might need a Data Engineer first.
It is well worth understanding the breadth of data, coding & project delivery skills that competent Data Engineers possess. A recent lunch with another data leader also brought to my attention the growing use of the term DataOps. For many years CIOs have seen the benefit of specialisms, with DevOps teams being particularly valued.
At a high-level (for my understanding) DataOps teams provide the much-needed mix of coding, data, technology & agile working skills that Data Science teams need to breakthrough their data barriers. With a focus on data & product implementation.
For more on DataOps, I recommend reading the recently published “Practical DataOps” from MoneySupermarket’s CDO, Harvinder Atwal. He has really opened my eyes to the need for such a team.
So, if your key business need is to overcome your data challenges, to overcome the time wasted while your Data Scientists wait for clean data – is it worth considering a Data Engineer rather than a Data Scientist?
Reasons to hire a Statistician
Of all the job titles that have gone out of fashion, near the top of that list must be a Statistician. I still recall when this was a highly respected technical job title. Organisations paid highly for qualified & experienced statisticians (or modellers) to join their analytics teams.
But, with the rise in popularity of Data Science the tables have turned. Most posts on Data Science blogs appear to claim the same techniques I used to hear from statisticians as now being “Machine Learning algorithms”. So, you can understand why managers seek to hire Machine Learning Engineers or Data Scientists.
When chatting with some of the candidates we’ve placed, especially graduates who went straight from Data Science degrees into industry, some common themes emerge. I hear that the majority of their work is using traditional statistics, like regression models & significance testing. Having hoped they’d be pioneering use of new Deep Learning algorithms; this can lead to frustration. With hindsight, a Statistician could have delivered all this work for less.
So, if your key business need is improved marketing targeting or measurement of business improvements – is it worth considering a Statistician rather than a Data Scientist?
Reasons to still hire a Data Scientist
Having said all the above, I want to still speak up for Data Scientists. Not only am I still hugely impressed by their breath of technical skills, I still see real needs for them in businesses. The job market may be overselling the need for this role, but when they are the right fit they can make a huge difference.
Clients who have made effective use of Data Scientists talk about the increased scientific rigour it has brought to their business. It sounds like part of the key is a willingness to change the way you work. Data Scientists work best when they have freedom to explore, experiment & refine, without the controls many businesses are used to.
Whilst that may sound like a recipe for chaos, I hear of business that have greater measurement & control as a result. Risking the freedom for these scientists to test assumptions & smash some sacred cows can lead to leaders actually understanding their business & customers better than ever. But it’s not a ride for the faint hearted.
So, if your key business need is innovation and improvement and you are willing to break your current rules to get there – then a Data Scientist may be the right hire for you.
Just remember our previous advice, that you need the right mix of roles in a Data Science team. Don’t focus solely on this one hire.
Do you need help with navigating this new world of job titles?
I hope those thoughts have helped you. At MBN we want to encourage the wider Data Science community to keep developing. But we also want to help hiring mangers think through what they really need. No one is helped by a poor fit with business need & we want to ensure the next generation of Data Talent get off to a productive start in their careers.
We’d also love to hear your experience. Have you decided against a Data Scientist and seen the benefit of hiring other roles? If so, please let me know, as we all have much to learn if we are to create the thriving data careers of the future.
If you’d value some help with which roles to hire for your business, or a chance to bounce your ideas off people who are close to the market, please get in touch – michael@mbnsolutions.com