Aleph Alpha has released Kolibri, an English-German Mixture-of-Experts model with open weights licensed under Apache 2.0.
Kolibri features 78 billion total parameters, with 3 billion active, and supports context lengths up to 1 million tokens. Developed through a refined training pipeline, it allows for stable pre-training without manual intervention during hardware or connection issues.