Meta’s Open Source Llama Upsets the AI Horse Race

Meta’s license for Llama 2 also requires companies with more than 700 million monthly active users to establish a separate license agreement with Meta. It is not clear why, but the clause creates a barrier to other tech giants building on the system. The model also comes with an acceptable use policy, which prohibits generating malicious code, promoting violence, or enabling criminal activity, abuse, or harassment. Meta did not respond to a question about what actions it might take if Llama 2 was used in breach of that policy.

Jon Turow, an investor at Madrona Ventures in Seattle, says Meta’s pivot from trying to restrict distribution of the first Llama model to open-sourcing the second could enable a new wave of creativity using large language models. “Developers and entrepreneurs are very resourceful, and they are going to find out what they can squeeze out of Llama 2,” he says.

Turow likens Meta’s choice to release Llama 2 this month to Google introducing the Android mobile operating system in 2007 to rival Apple’s iOS. By giving away a cheap but powerful alternative, Meta can become a counterbalance to proprietary systems like the kind developed by OpenAI, sparking innovation that could feed back ideas that help improve Meta products and services.

Llama 2 is the first openly released model on par with ChatGPT, says Nathan Lambert, an AI researcher at Hugging Face, a startup that releases open source machine-learning software, including generative models. He doesn’t consider the project truly open source, because of Meta’s limited disclosures about its development, but he is astonished by the number of Llama 2 variations he sees in his social media feed. One example is the latest version of WizardLM, an AI system, similar to ChatGPT, designed to follow complex instructions. Eight out of 10 models trending currently on Hugging Face, a number of which are made to generate conversational text, are variations of Llama 2.

“I think there’s a case to be made that Llama 2 is the biggest event of the year in AI,” Lambert says. He says proprietary models have the advantage today, but he believes that later versions of Llama will catch up and, before long, will be able to perform most tasks that people turn to ChatGPT for today.

Lambert also says the Llama 2 release leaves a number of questions unanswered, in part due to the lack of documentation of training data. And it will still remain the case that only major players like Meta, Google, Microsoft, and OpenAI will have the computing resources and staff needed to make leading large language models.

But he is hopeful that, despite the the success of OpenAI’s proprietary approach, language models are shifting into an era of transparency. A voluntary agreement  between the White House and seven major AI companies calls for tests of things like potential for discrimination or impact to society or national security before deployment. 

It’s a trend that could be challenged by growing questions about legal liability for AI systems and increasing regulatory pressure from politicians, who fear that malicious actors will start using open source models.

Like Demis Hassabis, the AI researcher now leading Google’s AI development, Turow disagrees with the assertion made by the leaked Google memo that it and other major AI companies are threatened by open source AI. He thinks data, talent, and access to computing power will continue to protect the biggest tech companies—but not make them invincible.

He’s now watching to see what startups and researchers do with Llama 2, expecting to see them rapidly improve it, as happened with the first iteration of Meta’s model. He says that should create new possibilities for both startups and the broader field of AI. “We’re seeing open source continually get better and better, so there may be surprises that upset the early leaders,” Turow says. “I don’t know what will happen.”