The connection between machine learning and human learning

What don’t you know that you need to know, and how do you know you don’t know? Imagine putting that into a search engine, expecting a coherent answer. The answers we seek are at the core of discovery and learning, motivated by necessity and pleasure, by job displacement, or leadership uncertainty in a complex and fast changing world. The process of learning requires a detailed knowledge of ourselves and of our world, whether a human or a machine is tasked to help.

We can fill gaps in our skills and knowledge through web searches, discussion with experts and like-minded people, reading books, working through an education curriculum, learning online with video tutorials, and absorbing a vast amount of content flowing through online news channels and aggregators. Some of this is structured learning, but a lot of it is being receptive to serendipity whilst stumbling around in the dark.

Machine Learning is now moving in to help. It is absorbing and modeling what you know and don’t know, to enable more targeted information to help individuals learn. This is where platform companies—like Google, LinkedIn, and Amazon are headed – building algorithms that represent us and our needs in data. Lifelong learning will become powered by platform-driven matchmaking technologies, which will know enough about us to make recommendations, and adapt to our changing contexts. This is the connection point between human learning and machine learning.

At the heart of this opportunity is data, a lot of it, and algorithms which know how to convert it into a matrix of comparisons for matchmaking. Amazon has long used this approach to recommend products. “Customers who bought this item also bought…”. On Tinder, a matchmaking algorithm compares the data in one profile against others, to supply a list of opportunities. On Facebook, the matchmaking is used to target advertising. Effectively, your data is becoming a representation of you, with a specific purpose in mind. The algorithm needs to know enough about you and what you are looking for to be able to supply options in ranked order. This technique is simple, ubiquitous and surprisingly powerful. Yet, the utility of these algorithms is way beyond which book to read next, and it is only just beginning.

LinkedIn’s Economic Graph is “digitally mapping the global economy to connect talent with opportunity at massive scale.” It gives them great insight into the changing nature of work, the types of skills that comprise specific jobs, and the work-related behaviours that pre-signal a change of job. It can also recognise a skills gap that needs to be filled before being able to move forwards in a career. Recognising people’s skills gaps, and helping to fill them, makes perfect sense of their purchase of the content aggregation app Pulse in 2013, of the online learning site Lynda.com in 2015, and the launch of LinkedIn Learning in September 2016.

One irony of the next few decades is that Machine Learning will increasingly take away knowledge jobs such as contract law and secretarial work, but it will also identify and recommend learning opportunities to help those people fill newly created gaps in their own knowledge in order to get new jobs.

In the coming years, the same technologies that power these innovations will become widespread for all sorts of companies. Data-driven companies and culture have been spoken about for some time, but in reality it is about becoming knowledge-driven. Departments and teams work in silos, with what they know locked away in their heads; internal processes are not optimised; staff training can be close to non-existent; the company’s board may have little insight into internal dynamics, and the CEO may not have learnt about disruptive threats hurtling towards the company’s bottom line.

Discontent, transparency break down, and sub-optimal use of talent within organisations can all benefit from the same kind of recommendation algorithms as those used to choose music. “Colleagues who solved this problem also spoke to…”; “CEOs who lacked this know-how also lacked …”; “Businesses that needed this service also needed…”. You can see where this is headed, even with a little tongue in cheek.

Despite the current explosion of interest around machine learning and artificial intelligence, we are as far from understanding our personal learning journeys as we were from understanding that the Earth rotates around the Sun before Copernicus. A knowledge-based revolution, powered by machine learning, is going to hit the business world and bring with it a paradigm shift for how companies structure themselves and interoperate. Businesses will utilise new and commoditised technology-driven techniques to learn, optimise and evolve.

Whilst machine-learning-based recommendations are driving revenue for web platforms and ecommerce websites, they are still far from sophisticated. They are surprisingly successful, even though they are mostly only using loose statistical methods to compare matrices of information that then supply a set of options. This is a grid, not a decision.

The weakest link in these machine learning systems can often be the coders and their companies, bringing bias through poor algorithm modeling and shortcut thinking. Buying a pair of shoes on Amazon can easily lead to the very same shoes being advertised for weeks. The algorithm seems not to know what has already been bought. There is also the problem of personal data pollution—which can occur if you buy a book for a parent or friend. Suddenly the algorithm thinks this is a preference of yours. Also, a filter bubble of popularity can keep articles at the top of a reading list through a self-confirming loop of people clicking the top of the list. Advertising funding elevates the priority of one item over another. Currently we are witnessing the issue of fake news, undermining people’s perception during elections. Social velocity easily becomes a distorted signal in the recommendation algorithm, and many people have few checks-and-balances for how to sieve facts from falsehoods.

This represents an opportunity for relevance engineering. Our learning algorithms are under-evolved. We still mostly operate with principles of similarity—this is like that, so have some more. Skill dependencies and complementary relationships between knowledge domains will need to be modelled for the algorithms to meaningfully inform personal learning experiences.

In 2015, TED released watson.ted.com as a collaboration between them and IBM Watson. The full archive of TED talks has been catalogued and fed into Watson. The insights locked away inside thousands of hours of videos had been freed up for Machine Learning to analyse. Speech-to-text services converted the videos into the written word for Natural Language Processing to then extract concepts, people, places, and things, and classify them against topics.

This process is one of creating a knowledge structure from previously unstructured words in transcripts. Content has become data. This is equivalent to the Machine Learning having read every book that you have read, and all the books that you haven’t, and making a recommendation based on comparing all the content. That’s a big calculation. Log in to TED with your Facebook ID and it will perform a similar process on your history of social posts. It will come to know you and the people that you know quite intimately and it will be able to compile a highly tailored list of videos for your personal learning pleasure. This is way more powerful, useful, and potentially accurate.

Note: The image is a visualization of a Stanford project, taken from the article Semi-automatic method for grading a million homework assignments at O’Reilly Radar. The overview of the method used to visualise ~40,000 student submissions to Coursera’s Machine Learning course is on Jon Huang's site.

The risk vs. reward of this kind of approach is clear. At a time when content channels try to maintain a competitive edge by flooding the market with volume, people are overwhelmed. A survey by The Economist verified that this phenomenon has caused confusion in three fifths of global executives, and that thought leadership content is so overcrowded that it has devalued itself. Knowledge flow is broken because volume no longer works. People need networked knowledge, personalised knowledge and learning filters. Currently, this can only be achieved via a highly select number of content channels to consume. In the future it will be enabled by a tailored algorithm that knows you. Give a little of yourself in data, and get a lot back in terms of personalisation—as long as trust is in place to use your data respectfully.

The World Economic Forum’s Future of Jobs report highlights that, between 2015 and 2020, 35% of core skills will change. Likewise, a Forrester report expects robots to replace 25% of automatable jobs within four years. This disruption will become increasingly visible. For example, the self-service check-out-free Amazon Go store was launched in Seattle in December 2016; Uber has taxis on the streets of San Francisco, ready to go fully driverless, but waiting for DMV approval. These are not necessarily humanoid-looking robots but are machines that can sense the world and work to the script of replicable processes.

With so much change around us, we will increasingly want learning to be baked into the rhythm of our lives, with machine learning acting as the connective glue for our topics of interest and comprehension levels. As we move further into an age where machine learning augments us, we will want our learning data to be multi-channelled and portable, with increasingly more sophisticated recommendation services, that matchmake us with our learning needs.

We will still want to meet experts at conferences to exchange niche ideas. We will still want our attention focused by a structured curriculum, with evaluation criteria and certification. And we will still want to share ideas within a group and feel that we are within a community of learners and teachers. The early signs of this are readily visible on edX, Coursera, Khan Academy, Lynda.com and other Massive Open Online Courses (MOOCs), with their peer review, discussion boards, auto-marking, interactive exercises, and learning analytics. Udacity even offers corporate courses and Nanodegrees to get Silicon Valley Skills for the jobs of tomorrow.

Lifelong Machine Learning is our future. Through our personal data, we will have the opportunity to train our own teachers and become hybrid learners. The more bespoken this becomes, the more human-centred it becomes. Machines will discover what you don’t know and what you need to know to compete and succeed, through prompting and finding patterns in the data. Networked and interconnected knowledge will bring new insights, opportunities and, perhaps, even new ways to learn.

The views expressed in this article are those of the author alone.

This article was originally published on E 180 Magazine.