Why you need to understand machine learning

Algorithms lie at the heart of machine learning, which, in turn lies at the heart of much of modern life–from online shopping to intelligence gathering. But most of us know little about these powerful tools and how they work. Is this wise?

You may not know it, but machine learning is all around you. When you type a query into a search engine, it’s how the engine figures out which results to show you–and which ads. When you read your email, you don’t see most of the spam because machine learning has filtered it out. Go to Amazon to buy a book or Netflix to watch a video, and a machine-learning system helpfully recommends some others you might like. Facebook uses machine learning to decide which updates to show you, and Twitter does the same for tweets. Whenever you use a computer, chances are machine learning is involved somewhere.

Traditionally, the only way to get a computer to do something– from adding two numbers together to flying an aeroplane–was to write down an algorithm explaining how, in painstaking detail. But machine-learning algorithms are different: they figure it out on their own, by making inferences from data. And the more data they have, the better they get. Now we don’t have to programme computers; they programme themselves.

It’s not just in cyberspace, either. Machine learning plays a part in every stage of our lives: grading college applicants’ exams, screening job candidates, picking stocks, diagnosing illnesses, finding partners. Society is changing, one learning algorithm at a time. Machine learning is remaking science, technology, business, politics, and war. Satellites, DNA sequencers, and particle accelerators probe nature in ever finer detail, and learning algorithms turn the torrents of data into new scientific knowledge. Companies know their customers like never before, and political candidates know their supporters. Unmanned vehicles pilot themselves across land, sea and air. No one programmed your tastes into Amazon’s recommendation system–it used your purchasing history to figure them out on its own. Google’s self-driving car taught itself how to stay on the road; no engineer wrote an algorithm instructing it how to get from A to B.

Machine learning is something new under the sun–a technology that builds itself. Ever since our remote ancestors started sharpening stones into tools, it is only humans who have designed artefacts. But learning algorithms are artefacts that design other artefacts. A learning algorithm is like a master craftsman: every one of its productions is different, and exquisitely matched to the customer’s needs. But instead of turning stone into masonry or gold into jewellery, machine learning turns data into algorithms. And the more data it has, the more intricate the algorithms.

Humans are the species that adapts the world to itself instead of adapting itself to the world. Machine learning is the newest chapter in this million-year saga: with it, the world senses what you want and changes accordingly, without you having to lift a finger. Like a magic forest, your surroundings–virtual today, physical tomorrow–rearrange themselves as you move through them. The path you picked out between the trees and bushes grows into a road. Signs pointing the way spring up in the places you got lost.

These seemingly magical technologies work because, at its core, machine learning is about prediction–predicting what we want, the results of our actions, how to achieve our goals, how the world will change. Once upon a time, we relied on shamans and soothsayers for this, but they were much too fallible. Science’s predictions are more trustworthy, but they are limited to what we can systematically observe and tractably model. Big data and machine learning greatly expand that scope. Some everyday things can be predicted by the unaided mind, from catching a ball to carrying on a conversation. Some things, try as we might, are just unpredictable. For the vast middle ground between the two, there’s machine learning.

Paradoxically, even as they open new windows on nature and human behaviour, learning algorithms themselves have remained shrouded in mystery. Hardly a day goes by without a story in the media involving machine learning, whether it’s a tech company launching a virtual personal assistant; Google’s AlphaGo beating the human Go champion; US retailer Target finding out a teenager is pregnant before her parents do; or the US National Security Agency (NSA) looking for dots to connect. But in each case the learning algorithm driving the story is a black box. Even books on big data skirt around what really happens when the computer swallows all those terabytes and magically comes up with new insights. At best, we’re left with the impression that learning algorithms just find correlations between pairs of events, such as Googling “flu medicine” and having the flu. But finding correlations is to machine learning no more than bricks are to houses, and people don’t live in bricks.

When a new technology is as pervasive and game-changing as machine learning, it’s not wise to let it remain a black box. Opacity opens the door to error and misuse. Amazon’s algorithm, more than any one person, determines what books are read in the world today. The NSA’s algorithms decide who is, or isn’t, a potential terrorist. Climate models decide what’s a safe level of carbon dioxide in the atmosphere. Stock-picking models drive the economy more than most of us do. You can’t control what you don’t understand, and that’s why you need to understand machine learning–as a citizen, a professional, and a human being engaged in the pursuit of happiness.