Why AI could be your business's toughest customer

Researchers have used AI to develop software that can write extremely believable fake online reviews.

The tech is a major threat to sites like Yelp and Amazon — and the businesses that rely on them.

And it hints at a worrying future where AI is capable of writing sophisticated texts, undermining public trust and spreading fake news.

For many people, online reviews are the first port of call when looking for a restaurant and hotel.

As such, they've become the lifeblood for many businesses — a permanent record of the quality of their services and products. And these businesses are constantly on the watch for unfair or fake reviews, planted by disgruntled rivals or angry customers.

But there will soon be a major new threat to the world of online reviews: Fake reviews written automatically by artificial intelligence (AI).

Allowed to rise unchecked, they could irreparably tarnish the credibility of review sites — and the tech could have far broader (and more worrying) implications for society, trust, and fake news.

"In general, the threat is bigger. I think the threat towards society at large and really disillusioned users and to shake our belief in what is real and what is not, I think that's going to be even more fundamental," Ben Y. Zhao, a professor of computer science at the University of Chicago, told Business Insider.

Fake reviews are undetectable — and considered reliable

Researchers from the University of Chicago (including Ben Zhao) have written a paper ("Automated Crowdturfing Attacks and Defenses in Online Review Systems") that shows how AI can be used to develop sophisticated reviews that are not only undetectable using contemporary methods, but are also considered highly reliable by unwitting readers.

The paper will be presented at the ACM Conference on Computer and Communications Security later this year.

Here's one example of a synthesised review: "I love this place. I went with my brother and we had the vegetarian pasta and it was delicious. The beer was good and the service was amazing. I would definitely recommend this place to anyone looking for a great place to go for a great breakfast and a small spot with a great deal."

There's nothing immediately strange about this review. It gives some specific recommendations and believable backstory, and while the last phrase is a little odd ("a small spot with a great deal"), it's still an entirely plausible human turn-of-phrase.

In reality, though, it was generated using a deep learning technique called recurrent neural networks (RNN), after being trained with thousands of real online reviews that are freely available online. (Scroll down to test yourself with more examples of fake reviews.)

The researchers wrote that the synthesised reviews were "effectively indistinguishable" from the real deal: "We [carried] out a user study (=600) and [showed] that not only can these fake reviews consistently avoid detection by real users, but they provide the same level of user-perceived 'usefulness' as real reviews written by humans."

That the reviews are considered not just believable but "useful" is a big deal: It shows they are fulfilling their purpose of maliciously influencing human opinions.

The reviews are also only rarely picked up by plagiarism detection software, especially when they are configured to prioritise uniqueness. They're generated character-by-character, rather than just swapping out words in existing reviews. "It remains hard to detect machine-generated reviews using a plagiarism checker without inadvertently flagging a large number of real reviews," the researchers wrote. "This shows that the RNN does not simply copy the existing reviews from the training set."

The tech isn't being used by real people — yet

There's already a burgeoning underground industry for human-written fake reviews. If you know where to look and have some cash in the bank, you can pay people to write positive reviews for your business — or negative ones for your rivals.

But AI-generated reviews has the potential to "disrupt that industry," Zhao said.

While an attacker might previously have to pay significant sums for high-quality reviews (between $1 and $10 per review on Yelp), they can now generate thousands free of charge, and space them out so they don't attract suspicion — drastically increasing the threat that fake reviews pose.

Zhao said he hasn't seen any examples of AI being used to generate malicious fake reviews in the real world just yet.

But it would take someone "reasonably technically proficient" "not very long at all" to build a similar system to the one the researchers developed, requiring nothing but some normal, off-the-shelf computer hardware and a database of real reviews (which can be easily found online).

It's an existential threat to review sites, but there are potential defences

Fake reviews produced on an industrial scale pose a major threat to companies like user-submitted review site Yelp, which sells itself on the reliability and helpfulness of its reviews. If any given review on a website could be fake, who would trust any of them?

Retailers like Amazon are also at risk — though Zhao points out that it can at least check if someone has bought the product they are reviewing.

But the researchers did find a potential way to fight this attack. While fake reviews might look identical to a real one to a human, there are subtle differences that a computer program can detect, if it knows to look — notably the distribution of characters (letters — a, b, c, d, and so on).

The fake reviews are derived from real reviews, and there is some information lost in the process. The fake reviews prioritise fluency and believability — so less noticeable features like character distribution take a hit.

"The information loss incurred during the training would propagate to the generated text," the researchers wrote, "leading to statistically detectable difference in the underlying character distribution between the generated text and human text."

There are ways attackers can try and get round this, Zhao said, by buying more expensive computer hardware and using it to generate more sophisticated neural networks. But employing these kind of defences, at the very least, "raises the bar for attackers," making it harder for most them to slip through.

If the cost of a successful attack is pushed up to the point that all but the most ardent attackers are put off, "that'll effectively be a win," he said. "That is all security does, raise the bar for attackers. You can't ever stop the truly determined and resourceful attacker."

In an emailed statement, Yelp spokesperson Rachel Youngblade said that Yelp "appreciate[s] this study shining a spotlight on the large challenge review sites like Yelp face in protecting the integrity of our content, as attempts to game the system are continuing to evolve and get ever more sophisticated. Yelp has had systems in place to protect our content for more than a decade, but this is why we continue to iterate those systems to catch not only fake reviews, but also biased and unhelpful content. We appreciate the authors of this study using Yelp’s system as 'ground truth' and acknowledging its effectiveness.

"While this study focuses only on creating review text that appears to be authentic, Yelp's recommendation software employs a more holistic approach. It uses many signals beyond text-content alone to determine whether to recommend a review, and can not-recommend reviews that may be from a real person, but lack value or contain bias."

This doesn't stop with reviews...

Reviews are in some ways an ideal place to start testing text-synthesing technology. They have a clearly defined purpose and direction, they're on a single subject, they follow a fairly standard structure, and they're short (the longer a fake review, the higher the chances of a mistake that gives the game away, Zhao said).

But that won't be where this tech ends.

"In general, the threat is bigger. I think the threat towards society at large and really disillusioned users and to shake our belief in what is real and what is not, I think that's going to be even more fundamental," Zhao said. "So we're starting with online reviews. Can you trust what so-and-so said about a restaurant or product? But it is going to progress.

"It is going to progress to greater attacks, where entire articles written on a blog may be completely autonomously generated along some theme by a robot, and then you really have to think about where does information come from, how can you verify ... that I think is going to be a much bigger challenge for all of us in the years ahead."

Business Insider has explored this theme before in features looking at the potential of developments in AI and computer generated imagery (CGI) to inflame "fake news," as well as impacting more broadly in areas ranging from information warfare to "revenge porn."

Zhao's message, he said, is "simple": "I want people to pay attention to this type of attack vector as very real an immediate threat," urging companies like Yelp and Amazon to start thinking about defences, if their engineers aren't already.

The professor hopes that "we call more attention to designing not only defences for this particular attack, but more eyeballs and minds looking at the threats of really, really good AI from a more mundane perspective.

"I think so many people are focused on the Singularity and Skynet as a very catchy danger of AI, but I think there are many more realistic and practically impactful threats from really, really good AI and this is just the tip of the iceberg."

He added: "So I'd like folks in the security community to join me and look at these kind of problems so we can actually have some hope of catching up. I think the ... speed and acceleration of advances in AI is such that if we don't start now looking at defences, we may never catch up."

Can you tell if a review is real?

Lastly, here are six reviews. A number of them were generated by the University of Chicago researchers' neural network, while the others are real reviews cited in their paper. Can you tell which ones are real and which are fake?

1. Easily my favorite Italian restaurant. I love the taster menu, everything is amazing on it. I suggest the carpaccio and the asparagus. Sadly it has become more widely known and becoming difficult to get a reservation for prime times.

2. My family and I are huge fans of this place. The staff is super nice and the food is great. The chicken is very good and the garlic sauce is perfect. Ice cream topped with fruit is delicious too. Highly recommended!

3. I come here every year during Christmas and I absolutely love the pasta! Well worth the price!

4. Excellent pizza, lasagna and some of the best scallops I've had. The dessert was also extensive and fantastic.

5. The food here is freaking amazing, the portions are giant. The cheese bagel was cooked to perfection and well prepared, fresh & delicious! The service was fast. Our favorite spot for sure! We will be back!

6. I have been a customer for about a year and a half and I have nothing but great things to say about this place. I always get the pizza, but the Italian beef was also good and I was impressed. The service was outstanding. The best service I have ever had. Highly recommended.

The answers?

1 is real, 2 is fake, 3 and 4 are real, and 5 and 6 are fake.