Perfecting the art of the whole organisation with machine learning

How do you capture continuous insights on your business that can help you make more informed decisions? You wade through the data. Or you let artificial intelligence (AI) and machine learning (ML) do it for you.

Proliferation of data, ease of storage and computing power on the cloud, increased adoption of ML and the rise of auto-ML have made the desire for ML models pervasive in organisational decision making. But what happens when these carefully created models cannot be scaled easily to satisfy every department within an organisation? Or when deployed models become obsolete or ineffective due to ever-changing data patterns?

An ML algorithm does not exist in a vacuum or, more specifically, in a silo within a company, tended to by a group of brilliant technical savants writing code. At least it shouldn’t. ML models should be based on a factory approach with repeatable processes, automation techniques and appropriate checks and balances. That's how ML models will scale to achieve their full potential for a business with real-world problems to solve.

Here’s why companies are currently struggling with this issue:

Think about what might happen to airline ticket purchases over the course of a year. Whatever business model works in the summer, might not work in the winter. While flight demand surges during the holiday seasons, airlines may struggle to fill the same seats when school is in session. Close attention needs to be given to the models over time to ensure they aren’t having a negative effect on revenue.

Many of these challenges can be solved with tried-and-true development models that carry a few new provisions with them. MLOps builds on the core principles of DevOps – automation, deployment beyond one-time use, process that includes integration, testing and releasing, as well as infrastructure operations. It is not enough to focus on sophisticated model development. It is also imperative to focus on building foundations to processes that are reliable and repeatable. Applying MLOps takes a machine learning model out of the silo and into the entire organisation where it’s most useful. The model can’t be developed for development’s sake if it’s going to create value. The business has to engage with it and use it.

Scaling many models and sustaining them requires a team of people with capabilities in a wide range of expertise. On one end of the spectrum is the data scientist with expertise in computer science, modeling, statistics, analytics, and math. At the other end is the domain expert with expertise in the specific industry, the business function, and the end user. In between are ML engineers, cloud engineers, data engineers and visualization experts, all working towards the common goal of using data to transform an enterprise.

Implementing MLOps requires a team model that sits at the intersection of skills and process. It pulls together a team with a range of skills and relies on automation, workflows and systems to drive impact on a sustained basis. The discipline matters more than the work. At the end of the day, your organisation could have hundreds of models.

MLOps must also be designed for innovation and change, so it can react to market conditions swiftly. Think of the pandemic causing major shifts in consumer behavior and the need for organisations to respond to the pandemic’s effects. In 2020, in Greece, passengers arriving from out of the country were tested by an ML algorithm that was trained to allocate scarce tests for the SARS-CoV-2 virus by learning which passengers were likely to test positive. Not only did it markedly increase the efficiency of testing, it contributed to Greece’s ability to keep its borders open safely.

The biggest risk with ML is that it won’t be continuously updated, retrained and validated. If it isn’t, the model won’t be as relevant as it could be and ultimately won’t result in real business value. Creating a suitable ML model is an end-to-end process that considers the context of the problem at hand, the desired use cases for the product, the decision process of the model and the vision for the product.

Importantly, successful MLOps – a marriage of people, process, technology and operating models – reflects the possibilities of the Age of With, in which human-machine collaboration through next-gen assets and platforms predict what is possible and translate the insight into performance. This has applications for everything from life-sciences company working on vital drug development, to a restaurant chain that wants to transform its pricing, profitability, and customer experience, or even an institution working to protect people in a pandemic.