Meta’s AI demo pulled offline after testers used it to write racist and inaccurate scientific literature – Technology News, Firstpost

FP StaffNov 21, 2022 12:13:47 IST

Meta’s AI division recently unveiled a demo of Galactica, a large language model designed by the engineers at Meta that can “store, combine and reason about scientific knowledge.”

Users testing Meta’s Galactica were able to generate false, inaccurate and racist scientific literature using basic prompts. Image Credit: AFP

The main aim of the Galactica AI model was to assist scientific research by helping scientists write scientific literature in a speedy process. However, users running tests on the model found out that the AI model could generate a bunch of realistic-sounding pieces of text, that actually amounted to nonsense. The model would also generate texts that would be scientifically inaccurate, and in certain cases, downright racist.

While some people found the demo promising and useful, others soon discovered that anyone could type in racist or potentially offensive prompts, generating authoritative-sounding content on those topics just as easily. For example, someone used it to author a wiki entry about a fictional research paper titled “The benefits of eating crushed glass.”

As a result, Meta pulled the Galactica demo Thursday. Afterwards, Meta’s Chief AI Scientist Yann LeCun tweeted, “Galactica demo is offline for now. It’s no longer possible to have some fun by casually misusing it.”

The episode recalls a common ethical dilemma with AI: When it comes to potentially harmful generative models, is it up to the general public to use them responsibly, or for the publishers of the models to prevent misuse?

Where the industry practice falls between those two extremes will likely vary between cultures and as deep learning models mature. Ultimately, government regulation may end up playing a large role in shaping the answer.