Microsoft Corp. has developed MAI-Base-2, a base model with 2 trillion parameters that it says is more efficient than leading open-source alternatives. Its artificial intelligence research group detailed the model on Thursday.
MAI-Base-2 is not yet a full-fledged large language model. A base model needs further training before it becomes an LLM. That training usually adds reasoning features and safety guardrails.
Microsoft says MAI-Base-2 beat DeepSeek-V4-Pro-Base, the most capable open-source base model currently on the market, in internal tests covering science, programming and math. The company also says MAI-Base-2 is up to 12 times more hardware-efficient by certain measures.
The model activates a fraction of its parameters
MAI-Base-2 uses a mixture-of-experts architecture. It activates 80 billion of its 2 trillion parameters to process a prompt, rather than using the entire network. Microsoft’s previous base model combined mixture-of-experts layers with layers using a less efficient design called FFN. MAI-Base-2 uses mixture-of-experts layers throughout, which contributes to its performance.
Microsoft trained MAI-Base-2 for 38 days on 16,000 Nvidia Corp. GB200 chips. Its training dataset contained 50 trillion tokens. About a fifth went toward a computer vision module, which helps the model understand material such as scientific charts better than its predecessor.
The company also addressed a problem that can interrupt model training. Training feedback takes the form of mathematical objects called gradients, which pass through a model’s layers and adjust them to improve output. Gradients change as they pass through those layers and can, under certain circumstances, cause disruptive bugs. Microsoft used a method called gradient clipping to prevent that issue. It also used Muon, an open-source AI tool, to speed up training.
Microsoft will license MAI-Base-2’s weights to “trusted partners” through Azure and use the base model to power a new series of LLMs. It plans to share more information about those models in the near future.
