Mixture of experts
2 RohitAI articles on Mixture of experts: news, explainers and practical guides, newest first.
llama.cpp b11390: CUDA MoE Memory-Fault Fix and Upgrade Guide
llama.cpp b11390 corrects CUDA MoE buffer padding. Identify relevant deployments, pin the fix and validate real workloads without assuming a speedup.
Olmo-core 3: What Changed and How to Plan a MoE Training Migration
Ai2’s Olmo-core 3 adds a MoE training stack. Check benchmark limits, PyTorch extras, cluster topology and checkpoint behavior before migrating.