Ai2 released Olmo-core 3 on Thursday as an open training framework for large mixture-of-experts (MoE) language models. Researchers and builders can access the code, but this is not a new Olmo model or a chatbot people can use. Ai2’s largest reported tests measure how the training system handles scale, not the quality of a completed model.
That distinction matters when reading the trillion-parameter figures. They describe configurations Ai2 tested, while the next Olmo MoE model remains a plan. The available reporting does not independently verify Ai2’s performance results.
What is Olmo-core 3, and who can use it?
Olmo-core 3 is training infrastructure: software for assembling and running the process that produces a model. Ai2 says its code is available in the public Olmo-core repository. Public code gives researchers something they can inspect and adapt; it does not give them a ready-to-use model trained with this new stack.
An MoE model has a pool of components called experts. For each token — a unit of text the system processes — a routing mechanism sends work to selected experts rather than activating every expert. That can increase a model’s total parameter count, a measure of its adjustable values, without using all those parameters for every token. The model’s weights and the training state used to update them must be stored across GPUs, while routing data to experts across GPUs adds communication costs.
Ai2 has built sparse models before: it says OlmoE used 64 routed experts. It describes the existing Olmo 3 models, by contrast, as dense models, in which nearly all components are active for each token. Ai2 says its next Olmo generation will use an MoE design. That is a stated plan, not a release date or a demonstrated capability of a future model.
Making training code public does not supply the GPUs, memory or networking needed to reproduce a large run. The available material does not quantify those requirements or their costs.
How does Olmo-core 3 spread MoE training across GPUs?
Ai2 says its redesigned stack replaces an earlier approach based on fully sharded data parallelism, which repeatedly gathers and splits model weights across devices. The new approach uses distributed data parallelism while keeping experts on GPUs and sending training data to them. Ai2 says the new approach avoids repeated weight gathering.
Other pieces divide the work differently. Expert parallelism distributes experts among GPUs; pipeline parallelism places different model layers on different devices. A distributed optimizer spreads the training state used to update model weights. Together, Ai2 says, these techniques let it configure larger MoEs without requiring one GPU to hold every component and its training state.
The value of keeping most experts inactive for a given token is visible in one Ai2 benchmark. With four experts selected per token, increasing the pool from eight to 128 raised total capacity from 4.6 billion to 47 billion parameters. Ai2 reports that active parameters per token stayed around 3.2 billion and that processing throughput fell by less than 5%. Those figures show what Ai2 measured in its setup; the available reporting does not establish that an independent group reproduced them.
Ai2 also reports a controlled test of MXFP8, a lower-precision format for representing numbers during training. On four NVIDIA B300 GPUs, it says MXFP8 increased throughput about 21% against BF16, another numerical format, while peak active memory dropped from 103 GiB to 95 GiB. Ai2 says work was spread uniformly among experts in that test. The result does not establish the same gain for other workloads or hardware setups.
Free newsletter
Get the morning briefing
Start each day with the AI stories that matter and why — a short, free email from our newsroom.
What do Ai2’s largest benchmarks actually show?
Ai2 reports a 1.2-trillion-parameter configuration across 512 GPUs, with 58.36 billion parameters active per token and peak observed throughput of 858 TFLOP/s per GPU. TFLOP/s measures trillions of floating-point arithmetic operations per second. Ai2 says it used random routing for these system tests: the routing did not test whether training produced a useful model. The figure is evidence of reported system performance under those conditions, not of a fully trained, evaluated trillion-parameter Olmo.
A larger number needs an even narrower reading. Ai2 says a short test using DeepEP v2 reached a 2.38-trillion-parameter configuration, but explicitly says it was not a full training run or proof that the system could sustain training at that scale. Capacity during a short test is different from the resources and reliability required to finish training.
One preliminary speed comparison has a reporting discrepancy. Ai2 says that in a test across eight NVIDIA B300 GPUs, its 47-billion-parameter MoE reached per-GPU throughput of 52,000 tokens a second. It puts its earlier implementation at 19,400 tokens a second per GPU, making the new stack about 2.7 times as fast in that test. SiliconANGLE’s report instead identifies the 19,400 figure as an NVIDIA Megatron-Core result. The accounts do not establish a verified comparison with Megatron-Core, and neither supplies independent replication of the new stack’s speed.
For builders, the immediate offering is code to examine and use in their own training work, not a model whose responses can be assessed. Ai2 has not given a release date for its planned Olmo MoE model in the available material. Independent tests of the framework’s speed, memory use and ability to sustain a large training run would do more to show how far its reported results travel beyond Ai2’s setups.