Llama 4 moves to MoE, and runs into a benchmark row
Meta released Llama 4 Scout and Maverick, moving from the dense Transformers of the previous generation to a sparse mixture of experts: Scout at 109 billion parameters with 17 billion active, Maverick at 400 billion with the same 17 billion active. Both took images as well as text, and Scout claimed a then-unprecedented 10-million-token context window. A two-trillion-parameter Behemoth was described as still in training and was not released. The launch landed on a Saturday, ahead of Meta's own LlamaCon later that month, which struck observers as odd, and it was followed by a row over evaluation when the version that scored well on LMArena turned out not to be the same as the weights that shipped. For the company that had carried the open-weight banner, it read as a stumble at the moment the field was catching up.