However, computers will notice these changes, which are carefully designed to weaken the ability of AI models to name their images. If the AI model is trained on these types of images, its capabilities will begin to crumble. He will learn, for example, that cars are cows, or that cartoon art is an impressionist-style drawing.
Show key points
"In this way, for a simple human or robotic inspection, the image looks in accordance with the text," writes Peng Edwards of Ars Technica, and then adds: "But in the latent and hidden space of the model, the image has the characteristics of both the original concept and the concept of poison, which leads the model astray when trained on such data and data.
Because models are trained on huge datasets, identifying toxic images is a complex and time-consuming task for tech companies, and even just a few misleading samples can do harm. When the researchers inserted 50 poisoned images, which described the dogs as cats, into Stable Diffusion, the model began generating distorted images of dogs. After 100 samples, the model began producing images closer to cats than dogs. When 300 samples were input, almost no dog-like traits remained.
Recommend
Previously, the team released a similar tool called Glaze, which hides the artist's style from AI tools trying to analyze it. Nightshade will eventually be integrated within Glaze.
Finally, the researchers hope Nightshade will help give artists more strength as they confront artificial intelligence, told Ben Zhao, a computer scientist at the University of Chicago who led the Nightshade team, Elaine Feeley of Hyperallergic.
"I think there's now very little incentive for companies to change the way they used to operate — which used to mean that 'everything under the sun is ours, and there's nothing you can do about it.'" says Zhao. "I think what we're doing is kind of giving them more push on the moral front, and we'll see if that actually happens."
While Nightshade software can protect artist works that date back to newer models, it cannot retroactively protect artworks from older models. "The software runs at training time and destabilizes the model forever," Zhang tells Axios Ryan Heath. "Of course, model coaches can go back to an old model, but that makes it difficult for them to build new models.
As Zhao told MIT Technology Review, there is a possibility that Nightshade could be misused for harmful purposes. However, he continues, "a targeted attack will be difficult, as it will require thousands of poisoned samples in order to damage larger models that have been trained on billions of data samples."
Marianne Mazon, a researcher in modern and contemporary art at Charleston College, who also works in Rutgers University's Art and Artificial Intelligence Lab, says Nightshade is an important step in the fight to defend artists facing tech companies.
Marianne tells Hyerallergic, "Artists now have something they can do, and that's important, feeling helpless is not good." At the same time, Mazoon fears that Nightshae may not be a long-term solution, but rather believes that creators should continue to follow legislative procedures related to AI image generation, because corporate finances and the rapid development of AI technology could eventually make software like Nightshade obsolete.
Meanwhile, Nightshade's presence is a morale boost for some artists, such as Ottman Beverly, who told MIT Technology Review that she stopped publishing her artwork online after discovering that her work had been collected and imitated without her consent. Tools like Nightshade and Glaze made her feel good when sharing her work online again: "I'm really grateful that we have a tool that can help bring back power to artists in their own work," she says.
