Sequelograph › Topics › mixture of experts
Mixture of experts
2 records filed under this topic, newest first.
- Mistral previews trillion-parameter model, promises October weights The Idle Rack What if only 49 billion parameters worked per token, but a trillion still had to be on hand?
-
An open model ships trillions of weights
The Open Weight
What if a lab's entire memory lived inside one open model?
technology open weights mixture of experts long context