GitHub - huawei-csl/MACaQ: SOTA model-agnostic quantization for GGUF, local inference and beyond. Quantize and run text, image, video and audio models with llama.cpp, vLLM, Diffusers and PyTorch.

Huawei’s Computer Systems Lab has released MACaQ, a model-agnostic quantization tool for GGUF and local inference that supports text, image, video, and audio models across llama.cpp, vLLM, Diffusers, and PyTorch. The project is positioned as state-of-the-art quantization for running models locally and beyond, with the repository hosted on GitHub under huawei-csl. It matters because it offers a single quantization approach across multiple runtimes and modalities, potentially simplifying deployment of compressed models. Next, watch for adoption benchmarks and compatibility updates across the named inference frameworks.