MicrosoftReleased on Aug 23, 2024

Phi-3.5-MoE-instruct: Benchmarks, Pricing & Size

Phi-3.5-MoE-instruct is a language model from Microsoft, released in August 2024.

Phi-3.5-MoE-instruct is a mixture-of-experts model with ~42B total parameters (6.6B active) and a 128K context window. It excels at reasoning, math, coding, and multilingual tasks, outperforming larger dense models in many benchmarks. It

Phi-3.5-MoE-instruct model size

Phi-3.5-MoE-instruct has 60 billion parameters and was trained on 4.9 trillion tokens. See how it compares to other models in the same parameter range.

ParametersTraining tokens
60B
4.9Ttokens
82× tokens-to-params ratio
Large (30–80B)
60B
1B7B70B405B

Phi-3.5-MoE-instruct API

API access coming soon

Phi-3.5-MoE-instruct will be available through our gateway shortly.

Phi-3.5-MoE-instruct examples

Recent arena outputs from Phi-3.5-MoE-instruct, picked from the highest-ranked matchups.

Phi-3.5-MoE-instruct license

Phi-3.5-MoE-instruct is released under the MIT license, which permits commercial use, has 60.0B parameters.

License
MIT
Commercial use allowed
Parameters
60.0B

MIT License - allows commercial use

FAQ

Common questions about Phi-3.5-MoE-instruct.

When was Phi-3.5-MoE-instruct released?

Phi-3.5-MoE-instruct was released on August 23, 2024 by Microsoft. This is the official Phi-3.5-MoE-instruct release date tracked on LLM Stats.

Is Phi-3.5-MoE-instruct available via API?

Yes, Phi-3.5-MoE-instruct is available via API. See the official documentation for authentication and endpoint details.

How big is Phi-3.5-MoE-instruct?

Phi-3.5-MoE-instruct has 60 billion parameters. It was trained on 4.9 trillion tokens. It ships as an open-weight model, so you can download and run it on your own hardware.

Who created Phi-3.5-MoE-instruct?

Phi-3.5-MoE-instruct was created by Microsoft.

What is the license for Phi-3.5-MoE-instruct?

Phi-3.5-MoE-instruct is released under the MIT license. This is an open-source / open-weight license that permits self-hosting.

Where is the Phi-3.5-MoE-instruct paper or technical report?

Phi-3.5-MoE-instruct has a paper or technical report available at https://arxiv.org/abs/2404.14219. Use that source for architecture, training, release and evaluation details.