What eBay tells us about AI, language and trade

Machine learning and artificial intelligence offer the hope of greater business efficiency. And yet, researchers have found virtually no empirical evidence supporting the promised strides in labor productivity and economic activity. That is, until now.

The new paper, which analyzes data from online e-commerce site eBay, is among the earliest tangible signs that AI and machine learning are living up to their promise.

Anecdotes vs. reality

“There is plenty of anecdotal evidence that AI has exceeded humans in many areas, but there was not much causal empirical evidence,” says Meng Liu, visiting assistant professor of marketing at Olin Business School at Washington University in St. Louis.

She notes that there is evidence that AI correlates with economic growth. “There seems to be a discrepancy between what our intuition says about AI versus what is actually observed,” she says.

For example, aggregate productivity growth rates have been stagnating since the 2000s.

Liu and coauthor Xiang Hui, an assistant professor of marketing, cite 2017 research from MIT and the University of Chicago highlighting the paradox between high expectations and modest productivity results for artificial intelligence. That paper, accounting for slower economic activity, cited stagnant or declining numbers for productivity and median income while the new technology burst on the scene.

“Pointing to aggregate growth statistics, AI pessimists say it’s not really helping our productivity,” Hui adds.

“The problem is that it normally takes time for organizations to ramp up complementary innovations, be it organizational or technological, to harvest AI’s benefits. This is where our paper comes in. Let’s look at the question in a friction-free platform where they use an AI-based translation system.”

International trade boost

Using data from eBay, which managed more than $14 billion in trade across more than 200 countries in 2014, Hui and Liu demonstrate that a moderate improvement in the quality of language translation increased trade between countries on eBay 10.9 percent.

The paper contrasts trading results between buyers and sellers in the United States against those in countries that do not primarily speak English, including countries in Latin America, Europe, and Asia. The researchers looked at trade before and after eBay implemented a new AI-driven machine translation service in 2014. By some measures, that service improved translation quality by about 10 percent.

The researchers also compared their results to a measure of trade cost Hui had explored in earlier research. As distances decrease between trading partners on eBay, the cost of trade drops. “What we show is that the introduction of machine translation is equivalent to reducing distances between countries by 26.1 percent.”

The research team took two approaches to analyzing the trade data. First, they contrasted US exports to countries where the advanced translation was available against those where it was not. The graph above shows results after the technology was introduced—and the visible increase in US exports to countries where it was available.

Next, the research team mitigated the effect of other factors that could have increased trade (more marketing, for example) by examining how machine translation affected longer versus shorter product titles. The theory was that translating longer titles required greater cost and effort, but would yield greater payoff—meaning the benefit of introducing machine translation should be higher for these items.

Meanwhile, if eBay increased marketing, it would affect longer titles similarly to shorter ones. However, the graph below shows the greater effect on longer product titles.

“These comparisons suggest that the trade-hindering effect of language barriers is of first-order importance,” the researchers write. “Improved machine translation has made the eBay world significantly more connected.”

The authors note that since their research was completed, Google has rolled out an even more powerful language translation tool “that has significantly improved translation quality” and, based on their research, “the effect (of Google’s software) on cross-border trade could be large.”

The authors argue that the introduction of machine translation on eBay provides a clean experiment where we can measure impacts. But ultimately, AI’s effect will like be seen in almost all economic sectors.

As new systems come online, the authors write, “they will provide new opportunities to assess the economic impact of AI via natural experiments such as the one examined in this paper.”