AIFIRST ConvRot Quant method now in llama-cpp-turboquant
The brief
The ConvRot quantization method, which outperforms standard UDQ8_K_XL in accuracy benchmarks, has been integrated into the llama-cpp-turboquant fork.
Key points
- ConvRot applies a learned rotation to model weights before quantization, reducing the accuracy loss from low-bit formats.
- The fork is available on GitHub for users who want better quantization quality beyond standard GGUF methods.
Sources
- redditreddit.com