Meta Is Developing a New, More Powerful AI System as Technology Race Escalates
Parent of Facebook and Instagram wants artificial-intelligence system to be as capable as OpenAI’s most advanced model
By Deepa Seetharaman and Tom Dotan
Wall Street Journal
Sept. 10, 2023 5:01 pm ET
Meta expects to start training the new AI system, known as a large language model, in early 2024.
PHOTO: DAVID PAUL MORRIS/BLOOMBERG NEWS
---------------------------------
Meta Platforms is setting its sights on OpenAI.
The parent of Facebook and Instagram is working on a new artificial-intelligence system intended to be as powerful as the most advanced model offered by OpenAI, the Microsoft -backed startup that created ChatGPT, according to people familiar with the matter. Meta aims for its new AI model, which it hopes to be ready next year, to be several times more powerful than the one it released just two months ago, dubbed Llama 2.
The planned system, details of which could still change, would help other companies to build services that produce sophisticated text, analysis and other output. It is the work of a group formed early this year by Meta Chief Executive Mark Zuckerberg to accelerate development of so-called generative AI tools that can produce humanlike expressions. Meta expects to start training the new AI system, known as a large language model, in early 2024, some of the people said.
\Plans for the new model, which haven’t previously been reported, are part of Zuckerberg’s effort to assert Meta as a major force in the AI world after it fell behind rivals. Competition in the area has sharply intensified this year, spawning divergent views on everything from which business models are best to how the technology should be regulated.
The company is currently building up the data centers necessary for the job and acquiring more H100s, the most advanced of the Nvidia chips used for such AI training. While Meta joined with Microsoft to make Llama 2 available on Microsoft’s cloud-computing platform Azure, it plans to train the new model on its own infrastructure, some of the people said.
Zuckerushing for the new model, like Meta’s earlier AI offerings, to be open-sourced and thereforberg is pble freee availa for companies to build AI-powered tools.
Llama 2 was released in July. / PHOTO: ANDRE M. CHANG/ZUMA PRESS
----------------------------------
\
Zuckerberg will be among a group of top tech executives attending a summit organized by Senate Majority Leader Chuck Schumer (D., N.Y.) on Wednesday to discuss how to handle AI. Sam Altman, OpenAI’s CEO, and Sundar Pichai, Google’s CEO, will also be attending.
The model under development may not close the gap with Meta’s competitors.
Meta hopes it will be roughly as capable as GPT-4, which OpenAI launched in March. GPT-4 underpins OpenAI’s moneymaking initiatives such as the recently launched ChatGPT for Business tool, and the company has been courting others to build on top of the technology as it tries to cover the enormous costs for advanced AI models. Meta’s new model also likely would come out after the expected debut of Gemini, an advanced large language model being built by Google.
Meta’s open-sourced approach has certain advantages. Zuckerberg has championed open-source AI models, which are popular for their lower cost and adaptability.
There also are potential downsides to an open-source model of the power Meta aspires to, say some legal specialists. These include increased risks around use of potentially copyright-protected information and broader access to a tool whose enhanced strength can be used to generate and spread disinformation or other bad actions.
Meta’s lawyers have raised some of these concerns as part of their review of the company’s plans.
“You can’t easily predict what the system would do or its vulnerabilities—what some open source AI systems offer is a limited degree of transparency, reusability and extensibility,” said Sarah West, a former adviser to the Federal Trade Commission who is now managing director of the AI Now Institute, a research institute that has raised concerns about big companies’ control over AI.
Large language models generally get more powerful when trained on more data. The most powerful version of the Llama 2 model that Meta announced in July was trained on 70 billion parameters, a term for the variables in an AI system that is used to measure size. OpenAI hasn’t disclosed the size of GPT-4, but it is estimated to be roughly 20 times that size, at 1.5 trillion parameters. Some AI experts say there could be other methods to achieve GPT-4’s power without necessarily approaching its size.
Write to Deepa Seetharaman at deepa.seetharaman@wsj.com and Tom Dotan at tom.dotan@wsj.com
Meta Is Developing a New, More Powerful AI System as Technology Race Escalates - WSJ (archive.ph) |
