How to become a Machine Learning Engineer in 2020

 By 2020 Machine Learning Engineer Change the World Faster and Evolve with Technology

The title of “Machine Learning Engineer” is quickly becoming more popular and with that, there is significant interest from people trying to enter the Data Science field. What kind of career path is this and what skill set does a Machine Learning Engineer need to have? Is it possible to define steps to take in order to become a ML Engineer? Can you follow online training and get certified? I figured I’d write up my ideas on the state of the field and how viable it is for those looking to pursue a career in it.

The Machine Learning Engineer

Let’s get one point out of the way first. Some might look at the job title and expect it to be a Data Scientist who purely focuses on model building and that’s it. This is a big no no; if only because most ML Engineering work starts after the initial model is built. While it’s often part of the job, a Machine Learning Engineer does not purely build models. And honestly, that part will only take up 5 to 10% of the job.

Look at this image of all the components that are involved in the model ecosystem. The black square at the center? That’s the actual ML code.

So what kind of creature is the Machine Learning Engineer then and where does it fit into the grand scheme of things? I prefer to fall back on a part of Tomasz Dudek’s definition from 2018: 

A person called a machine learning engineer asserts that all production tasks are working properly in terms of actual execution and scheduling, abuses machine learning libraries to their extremes, often adding new functionalities. (They) ensure that data science code is maintainable, scalable and debuggable, automating and abstracting away different repeatable routines that are present in most machine learning tasks. They bring the best software development practices to the data science team and help them speed up their work…

 Tomasz Dudek in “But what is this “machine learning engineer” actually doing?”

Essentially a ML Engineer is then some kind of wizard that brings models to production in a sensible way, is able to improve the Data Scientist’s models and is also partly an architect who lays the road for the Data Science team. This sounds incredibly like some kind of senior engineering role, and yet it doesn’t have to be.

---

Common ML Engineering backgrounds.

Most of the other ML Engineers I’ve met fall into one of two categories. The first group is highly educated, with most having a master’s or even a PhD in Computer Science, Artificial Intelligence, Data Science or Software Engineering. Surprisingly many are relatively new grads, with 1–3 years of experience under their belt when they became ML Engineers. There’s also a second group that consists of more experienced developers that transitioned into this role from neighboring fields such as Software Engineering or Data Engineering, and of course Data Science.

This indicates that there is a level of proficiency needed to be a ML Engineer that could come from either of the two directions that make up the role. You could be a great software engineer, or a fantastic machine learning virtuoso. Maybe both! If you are one already, this might be the field for you. If you are not, it might be a viable direction to develop yourself towards.

But do not make the mistake that Software Engineers or Data Scientists automatically make good ML Engineers. I come from a software background myself and I can vouch that most ML concepts and APIs are absolutely alien to Software Engineers. I remember the intense struggles I had getting to know TensorFlow and Theano years ago. Even though I started coding in my teens I had never seen anything like it. The experience was humbling.

A beginner-level ML Engineer is not a beginner programmer. This is a journey that is always at least traveled with experience. Is it then impossible to land a ML Engineering job without experience or training?

Of course not. However, the odds are against you. It is far easier to get into this niche when you have a similar background. There is some light on the horizon, however.

Remember that back when Data Science started becoming popular we said the same thing about Data Scientists because the people doing Data Science at that time were some of the brightest and most highly-educated people in the world. Since then Data Science has become more accessible and in truth, nowadays you can be a great Data Scientist without needing a PhD. Whether the same will fully apply for ML Engineering I am not sure, but I hope that as our field matures the barriers to entry will become lower.                     

             Software Engineer + Data Science = Machine Learning

Data Science. Software Engineering. Probably some linear algebra too. These were the ingredients chosen to create the perfect ML Engineer. Whiteboard creation by author.

The toolbelt of the ML Engineer is not simply the lovechild of an intense affair between a Software Engineer’s IDE and a Data Scientist Jupyter Lab. It has many tools and techniques that are intrinsic to the field. Which brings me the next section…

---

The Machine Learning Engineer’s Skills

Skills lists become outdated soon after being written, and often take on a life of their own. And yet I am here to draft up a non-exhaustive list of skills and topics to study! The tool landscape is so broad that it’s unlikely any ML Engineer will have proficiency with every language, tool and concept out there. Please don’t look upon this as some kind of list of items you need to cross off on your ML Engineering journey like so many online resources will instruct you to. Rather, take note and look at these as themes within the ML Engineering field.

I’ll try to discuss concepts more than specific tools. That way most of this will remain relevant in a couple of months or years.

Data Science

• Python. Look into coding standards and some of the cool stuff in the recent versions of Python. Having a basic understanding of R is also useful and your Data Scientists will thank you for it.
• Statistics.
• Model optimization.
• Model validation.
• ML frameworks such as sci-kit learn
• Deep learning frameworks such as TensorFlow and PyTorch
• ML applications such as NLP, computer vision and time series analysis.
• Mathematics. Implicitly, you’ll use a lot of linear algebra and calculus.

The reason why I would take Python over R or any other language is mainly because of the production aspect. While you can do a lot with R it is often not supported as well as Python is. There’s also the time aspect that plays here: often it is far faster to productionalize code in Python than R.

Software engineering

• Experience outside of python in a second programming language, such as Java, C++, or JavaScript.
• Cloud offerings. More on that later.
• Distributed computing
• System design and software architecture
• Data Structures and Algorithms.
• Databases and the query languages that come with it.
• Containerization (e.g. Docker, KubeFlow)
• Functional programming concepts
• Design patterns
• Big O
• API development
• Version control: git
• Testing
• Project management. Probably the most underrated element in any SE curriculum.
• CI/CD
• MLOps

So how do you learn about all of these if not on the job? Courses and online training can be great but they won’t teach you how to do apply it in a real-life setting. For things like statistics it doesn’t matter, but for technical subjects knowing “about” it is only half of the mastery. It doesn’t take more than a quick glance at Reddit’s r/learnprogramming to see that there are many people struggling to make the jump from coding in the protected IDE in an online course to coding their own projects on their own machine.

My experience is that it might be better to get started on a project on your own to learn a new skill, and supplement your knowledge with online training when you already have some applied knowledge. Instead of all-in-one training programs there are many tutorials online to help you with that, from building your own clock or calculator to a complete web app. Be aware of any course that promises you can go from zero to hero in a couple of weeks or months.

Certifications are a similar beast. A certification can be particularly valuable if you’re in consulting and want to signal to clients that your skills meet certain standards. Having a certification that corresponds to a client’s tech stack immediately puts you at the front of the pack. However, a certification is worthless without the skills to back this up in the first place. Consider now that you can obtain many certifications without having to code for them and you’ll see where I’m headed. Often, the time spent getting a certification would be better spent just building applications.

That said, there are some certifications that do carry some merit for ML Engineers, particularly for cloud vendors. Often these require a couple of years of experience deploying applications on their respective platforms, but anyone can pay $100–300 and register for a certification examination. As of 2020, there are three cloud vendors worth mentioning: Azure (Microsoft), GCP (Google), and AWS (Amazon). Here’s a list of certifications they offer that are in the sphere of interest of the ML Engineer.

 Microsoft Azure:

Microsoft offers associate-level certification for both Data Scientists and AI Engineers, as well as about a dozen other certifications. Some certifications actually require multiple exams, but this is not (yet?) the case for both the Data Scientist and the AI Engineer cert. The certification topics are a little bit superficial, but the exam should not be underestimated.

• Microsoft Certified: Azure AI Fundamentals
• Azure Data Scientist Associate
• Azure AI Engineer Associate 

Google Cloud Platform:

Google is the challenger when it comes to cloud services and the state of their certification reflects that. At the moment the ML Engineer exam is in beta and no certifications have been awarded yet. The exam takes four (!) hours but is an incredibly comprehensive list of what a ML Engineer’s job is all about. Prior to this certification being introduced, some ML topics fell under the Data Engineer certification, so many ML Engineers, myself included, actually took the Data Engineering certification track.

You could also look at the Google Cloud Architect, Developer or DevOps certification, but these barely touch upon it and might add a little bit of noise on your resume that lines you up for different gigs. I say that as a certified Cloud Architect myself who learned this from experience. On the other hand, it could make your profile a little bit more appealing.

• Google Cloud Certified Professional Data Engineer
• Google Cloud Certified Professional Machine Learning Engineer (currently in beta)

 

A Reading List for the Machine Learning Engineer

 

Although this is by no means meant as a complete list, here are some resources that I feel would benefit those who want to break into this field.

Books:

• Clean code by Robert C. Martin
• Machine learning yearning by Andrew Ng
• Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville
• The Pragmatic Programmer by David Thomas and Andrew Hunt
• Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems by Martin Kleppman

Posts / blogs:

• How to read scientific papers by Christoph Schmidl
• Everything you need to know about becoming a self-taught ML Engineer by Jason Benn has some excellent motives.
• Become a Data Scientist in 2020 with these 10 resources by Rahul Agarwan will appeal to those looking for lists of online resources.
• Openai’s blog

Papers:

• Hidden Technical Debt in Machine Learning Systems
• Software Engineering for Machine Learning: A Case Study

Conferences, meetups and the ML Engineering Scene

Some people hate them, others love them. Some use them as a way to self-promote themselves to the max while other people share absolutely brilliant ideas. I like meetups for the simple reason they’ll allow you to cherry pick what kind of topics you want to learn about. Do you like Scala? AI in Healthcare? Meetups. More of a fan of Bayesian optimization? There’s probably a meetup for that. Most of the meetups have gone fully online due to Corona and I expect this to continue through to the first half of 2021 at least.

Protip: They’re great for networking if you’re looking to get an internship, job, or a mentor.

Other Careers

There are a couple of other careers that should be mentioned to those considering moving into this field.

• Data Engineering: Anything that touches data needs to be able to handle scale and complex transforms. You’re the one specialized in connecting various elements of the data pipeline. There is often a significantly higher demand for your services than there is for those of the Machine Learning Engineer.
• Data Scientist: Analysis, Storytelling, Statistics, Machine Learning and presenting it to the CEO. You got it all. Usually this job is more diverse and involves less programming, but it really depends where you end up — there are so many flavors of data science it is hard to define it as a single role and some data scientists run a complete Data & Analytics department by themselves.
• Cloud Engineering: Specialized in integrating different applications and moving workflows to the cloud, you’re pretty good friends with the Data and ML Engineers.

Conclusion

A Machine Learning Engineer has a broad range of topics to understand from both Machine Learning and Software Development. Courses and certifications don’t bring you there as of 2020. A formal training or experience in the field is still desirable, but I expect that it will become more accessible over time, similar to how Data Science became more open to newcomers. With that in mind, I feel that the best path for those looking to become ML Engineers without formal training would then be to enter Data Science or Software Engineering, and transfer from there while picking up the elements that make up ML Engineering.