How to tackle machine learning’s MLOps tooling mess

Though MLOps tooling is bound to get easier, there are simple steps you can take to get value from machine learning today.

We’ve been overcomplicating machine learning for years. Sometimes we confuse it with the over-hyped artificial intelligence, talking about replacing humans with robotic reasoning when really ML is about augmenting human intelligence with advanced pattern recognition. Or we burrow into deep learning when more basic SQL queries would get the job done. But perhaps the greatest problem with ML today is how incredibly complicated we make the tooling because, as Confetti AI co-founder Mihail Eric has posited, the ML “tooling landscape with constantly shifting responsibilities and new lines in the sand is especially hardest for newcomers to the field,” making it “a pretty rough time to be taking your first steps into MLOps.”

SEE: Artificial Intelligence Ethics Policy (TechRepublic Premium) 

Normally we look to tooling to make tech easier. MLOps is doing the opposite. What can be done?

It’s tooling all the way down

The problem, Eric argued, is that no one wants to be left out of the ML Gold Rush. Given the promise to solve billion-dollar problems with the next algorithm, billions of dollars are being spent to create new companies. Each of those companies wants to sell you a new model/feature/metric/etc., store (which is just an unnecessarily fancy way to say database). Indeed, according to the recently published 2022 Stanford AI Index report, private venture investment in ML (and related AI) grew to $93.5 billion in 2021, more than doubling its 2020 tally. In turn, we’re seeing more research, more students, more everything flowing into ML.

And more tooling. Lots and lots more tooling.

Along the way, Eric noted, “the entire field is still standardizing the best way to architect fully-fledged ML pipelines. Achieving consensus around best practices will be a 5-10+ year transformation easily.” In the meantime, expect a somewhat frothy, chaotic environment for MLOps.

In the meantime, take heart. Though companies like Google and Amazon (Eric worked on Alexa while at Amazon) set the tone for ML’s promise, “The truth is there are only a handful of super sophisticated AI-first enterprises with robust machine learning infrastructure in place to handle their petabytes of data,” Eric stressed. Most of us are ML newbies, in other words, whatever our LinkedIn profiles may say to the contrary.

Or maybe “newbie” is the wrong term. Eric described a long tail of organizations with “ML at reasonable scale” that may have “good-sized proprietary datasets (hundreds of gigabytes to terabytes)” but “are still early in their ML adoption.” For such companies, he went on, “They don’t even necessarily require these super-advanced, sub-millisecond latency, hyper-real-time pieces of infrastructure to start levelling up their machine learning.”

So what should they do to get started?

One step at a time

According to Eric, the key is to take a deep breath and … do less:

1. Engage more experienced individuals to help you consider the options, think through different tech and be a sounding board for “dumb” questions.
2. Think carefully about the problem you are trying to solve and the fundamental methodology needed to solve it, rather than getting too distracted by shiny tools or platforms.
3. Spend a lot of time building real systems so that you can experience first-hand the pain points that different tools address.

This last tip resonates with a recent suggestion Microsoft’s Scott Hanselman made about software development: if you want to better understand software you need to “Run real sites and scale them.” As Hanselman and Eric suggest, it’s in building that you run into all the rough edges systems (and systems of systems) still present. In the case of MLOps, doing so won’t somehow make the morass of tooling options easier to manage, but the practical experience with building ML-based systems helps to filter out noise from signal.

Beyond individual learning, Eric is optimistic that, over time, the messy MLOps tooling landscape will sort itself out. Both cloud hyperscalers and smaller companies like DataRobot, he reasoned, will build out end-to-end systems that will make the tooling more approachable for those who want to go all-in on one vendor.

In the meantime, Eric’s three principles can help. On that second principle, it’s worth remembering something that investor David Beyer once argued: “The dirty secret of machine learning … is so many problems could be solved by just applying simple regression analysis” or a handful of if/then statements. Put more bluntly: Sometimes we want to apply ML to problems better solved with SQL queries and “basic math.”

Disclosure: I work for MongoDB but the views expressed herein are mine.