MACaQ: Lower WikiText2 perplexity than MR-GPTQ and DuQuant++ on Llama-3-8B, faster quantization and a reported 64 minutes for 122B expert layers
AI-generated summary Huawei CSL released MACaQ to create deployable 4-bit models by processing each linear layer independently to reduce quantization time and required GPU memory. In paper experiments on Llama-3-8B using WikiText2, MACaQ achieved lower perplexity than MR-GPTQ and the DuQuant++ family.