AIFIRST You can now use MTP in GLM-Air
The brief
GLM-4.5-Air, a 106B mixture-of-experts model with only 12B active parameters, now supports Multi-Token Prediction in llama.cpp for a meaningful inference speed boost.
Key points
- The update makes it more practical for users who want a large MoE without requiring multiple high-end GPUs.
- MTP support is enabled via a configuration flag in the updated build.
Sources
- redditreddit.com