Meta Unveils Llama 4 with MoE Architecture
On Friday, April 4, 2025, Meta announced the release of Llama 4, a new generation of AI models featuring the Mixture of Experts (MoE) architecture. This announcement marked a significant advancement in AI capabilities, offering enhanced performance across a range of language and multimodal tasks.
The Llama 4 model family consists of three models: Scout, Maverick, and Behemoth. Both Scout and Maverick were released with Scout having 17 billion active parameters and a 10 million token context window, while Maverick boasts the same number of active parameters but features a much larger context window of 1 million tokens. These models are the first in the Llama series to utilize MoE architecture, supporting both text and image inputs with text outputs, and providing multilingual capabilities in 12 different languages.
Furthermore, these models have been trained on over 30 trillion tokens, doubling the training data used for their predecessors, Llama 3. Meta is maintaining an open approach, providing open weights but instituting a monthly user limit of 700 million before requiring a commercial license. However, usage is restricted in the European Union due to existing AI and privacy regulations.
Meta CEO Mark Zuckerberg emphasized the company’s ongoing commitment to open-source AI, aiming to foster innovation and collaboration across the industry. More details on the models and their specifications can be found in Meta’s official announcement.
Overall, it was a relatively quiet day for AI news aside from Meta’s major announcement, reflecting the industry’s anticipation of the Llama 4 release.