What is Nightshade – the new tool allowing artists to ‘poison’ AI models?

The word “nightshade” brings to mind the highly poisonous plant, deadly nightshade. But now it is also the name of a “data poisoning tool” helping artists fight back against unauthorized use of their work by artificial intelligence (AI).

Nightshade has been developed by computer scientists at the University of Chicago, who have published a paper on their work, Prompt-Specific Poisoning Attacks on Text-to-Image Generative Models.

Nightshade allows artists to change pixels in their work in a way that is invisible to the human eye. This misleads AI programs, which then believe the image shows one thing, whereas viewers see something different.

“Dogs become cats, cars become cows, and so forth,” the MIT Technology Review explains in its preview article on Nightshade.

This approach is known as “poisoning” data. It will lead to generative AI tools creating images that are messed up or don’t match what is being searched for.

AI tools extract data from billions of images online, so the more Nightshade is used, the more AI could become poisoned, MIT Technology Review adds.

AI companies disrespect artists’ copyright and intellectual property when they train their image-generating tools on artists’ work without permission, says Ben Zhao, the computer science professor at University of Chicago leading the Nightshade project.

He hopes Nightshade will deter this behaviour and give some power back to artists.

Artists, performers and record labels have filed lawsuits against AI companies including OpenAI, the creator of ChatGPT, for training their AI on artists’ material without permission, reports tech news site VentureBeat.

Nightshade could have an important impact if AI companies are forced to be more respectful of artists and their rights, for example by agreeing to pay out royalties, say the contributors to the MIT article.

Some AI companies have mechanisms for artists to opt out of their work being used to train generative AI. But artists argue that this isn’t enough, and that AI companies still have too much power.

Generative AI is one of the top technologies listed in the World Economic Forum’s Top 10 Emerging Technologies of 2023 report.

The report highlights the technologies expected to have a positive impact on society in the next three to five years. Generative AI ranks in second place.

“While generative AI is still currently focused on producing text, computer programming, images and sound, this technology could be applied to a range of purposes, including drug design, architecture and engineering,” the Forum says.

Generative AI applications, though, should build public trust by meeting agreed professional and ethical standards.