Will Machine Learning Engineers Exist in 10 Years?
Machine learning will transition to a commonplace part of every software engineer’s toolkit.
In every field, we get specialized roles in the early days, replaced by the commonplace role over time. It seems like this is another case of just that.
Machine learning engineer as a role is a consequence of the massive hype fueling buzzwords such as "artificial intelligfence" and "data science" in the enterprise. In the early days of machine learning, it was a very necessary role. And it commanded a nice little pay bump for many. But machine learning engineer has taken on many different personalities, depending on whom you ask.
The purists among us say a machine learning engineer is someone who takes models out of the laboratory and into production. They scale machine learning systems, turn reference implementations into production-ready software, and often cross over into data engineering. They’re typically strong programmers who also have some fundamental knowledge of the models they work with.
But this sounds a lot like a normal software engineer.
Ask some of the top tech companies what "machine learning engineer" means to them, and you might get 10 different answers from 10 survey participants. This should be unsurprising. This is a relatively young role, and the folks posting these jobs are managers, often of many decades who don’t have the time (or will) to understand the space.
Here are a few requirements from job listings from some of the top tech companies. Nnotice how vastly they differ.
This first one is spicy. Are you sure this isn’t a researcher? How is this a machine learning engineer?
- PhD in math, stats, operations research. Knowledge of R, SQL, and modern machine learning techniques.
This next one’s more on-brand. And it comes from the top, so it shouldn’t be a surprise.
- BS or MS in computer science. 1–5 years of work or academic experience in software development. Exposure to Computer Vision, NLP, etc. a plus.
And, finally, drilling down on your stereotypical ML Engineer posting.
- BS/MS in computer science. Three or more years building production machine learning systems and efficient code. Experience with big data a plus.
Some companies have started a new approach, and I think most will follow. The approach is to list a software engineering role with exposure to machine learning as a core requirement plus a few years of experience as a preferred qualification. Employers will take a preference for engineers with experience building and scaling systems, regardless of whether it was based on machine learning or some other technology.
The machine learning engineer is necessary as long as machine learning understanding is rare and has a high barrier to entry.
It’s my earnest belief that the role of machine learning engineer will be taken over entirely by the common software engineer. It will transition to a standard engineering role where the engineer will get a specification or reference implementation from someone upstream, turn it into production code, and ship and scale applications.
For now, much of many machine learning roles exist in this weird space where we’re attacking problems with machine learning that just haven’t been attacked before. By consequence, machine learning engineers are, in many cases, half researcher, half engineer. I’ve come across my fair share of machine learning engineers who play across the entire stack. I’ve come across others who have a more narrow skill set but spend more time reading new research papers and turning them into usable code.
We’re at a weird crossroads where we’re defining where the members of our teams fit into the puzzle.
Don't miss out on the latest technology delivered to your email monthly. Sign up for the Data Science and Digital Engineering newsletter. If you are not logged in, you will receive a confirmation email that you will need to click on to confirm you want to receive the newsletter.