This AI has been debating real humans - and doing a pretty good job

As long as there have been personal computers, there have been people yelling at them. And before that, angry couch potatoes argued with their TV sets.

Now artificial intelligence technology that IBM has been developing over the past six years can give the machines a chance to argue back. The company took the wraps off an AI system Monday that it calls Project Debater, staging a couple of debates between it and experienced human debaters that show how far its capabilities have come.

IBM, which has been staging in-house debates between humans and Project Debater for a few years, said the demonstration represented the first such public debate. While Debater scored points on marshaling ample evidence for its arguments in a short period of time, the answers sometimes rambled and the delivery at times fell flat. Still, the technology, which remains in its early stages, represents a step forward in conversational AI.

Debater is designed to digest a single debate question, then scan billions of sentences from documents ranging from Wikipedia to magazine and news articles to craft its arguments, rebuttals, and closing summaries. Debater spells out its arguments in the placid monotone that is familiar to users of Siri and Alexa.

The demonstration in San Francisco involved two debates against professional debaters who have been working with the project. The two topics at hand on this day: Whether to subsidize space exploration and whether to increase the use of telemedicine.

Both topics were chosen from a larger list of potential questions, and aside from a brief introduction, none of Debater’s arguments were pre-arranged, IBM said.

Debater and its human opponents each had four minutes to state their arguments, another four minutes for rebuttals, and two minutes for their closing statements. In contrast to a warm body standing behind a dais, Debater was represented by a black obelisk the height of a human with three dots that indicated when it was listening or preparing an argument.

The program hewed tightly to the debate format, outlining arguments and responding directly to points its human opponents made with facts drawn from scientific research and world events. At one point, Debater accused an opponent—with some accuracy, it appears—of getting his facts wrong. While such a move is common in political debates, it carried a certain extra weight coming from an AI system.

Such moves, along with the attempts at humor, were part of Debater’s design to approximate the dynamics human debates. It was also able to anticipate potential counterarguments and rebut them ahead of time, a classic debate strategy.

For nearly 70 years, AI systems have developed in part by learning to play games such as checkers and chess. In 2011, IBM’s Watson won Jeopardy A few years later, Google’s Alpha Go defeated the top-ranked champion of the board game Go.

After Watson’s Jeopardy! win, IBM researchers took stock of its AI technology to determine what its next challenge would be. Noam Slonim, a researcher in IBM’s Haifa lab in Israel suggested an AI system that could debate a human being. The challenges of such a project were daunting.

“The challenge was a departure from the tradition of games where, even as complex as they may be, in the end you have a self-contained universe. There’s a set of rules,” Dario Gil, vice president of AI at IBM, told Fortune. “The thought of bringing AI into an area where, by its very definition, the problem is unbound was really intriguing. The question was, is it even possible?”

In 2012, Slonim and other researchers began working on Debater with an early prototype working by October of the next year. To succeed, Debater needed to accomplish three basic tasks, each of which marked new ground in AI: identifying key concepts and claims in spoken language; digesting a massive amounts of documentation and extract a clear argument; and modeling human dilemmas and controversies to create principled arguments.

For example, Gil said, if Debater is preparing a debate in favor of vegetarianism, it may extract an argument from something not explicitly related to that topic, such as a scientific study that suggests animals can feel fear and other emotions. The AI system would also need to eliminate redundant words and present its argument in a manner that could be persuasive to humans.

One IBM’s chief goals with Debater was to help an AI system master human languages. Speech dictation programs like Dragon passively transcribe language between text and speech without analysis. Voice-powered AI assistants like Alexa, meanwhile, engage in limited interactions that typically last a few seconds at time. But Debater was built to comprehend an argument that can last for several minutes, and then respond with its own.

In time, IBM plans to commercialize Debater, Gil said. The technology could have applications in a variety of fields, from education to law, and from government to businesses. Sales teams and trial attorneys could sharpen their arguments through an AI system capable of debating, while researchers and decision-makers could benefit from the pro and con arguments that Debater could generate.

Another potential application may be addressing fake news by examining how much, if any, evidence supports bogus claims and disclosing how much credibility they carry, Slonim said. “There’s no doubt the underlying tech we are developing will be valuable against fake news,” he said.

Beyond that, IBM hopes to keep developing the AI system powering Project Debater to tackle even more ambitious achievements.

“If you look at the horizon, you’re going beyond just information retrieval to a paradigm where you’re entering true reasoning – composing language that could give answers that we’ve never seen before” Gil said. “For AI to manifest itself and truly realize its potential, it’s going to have to live in the messy world where humans live.”