devotd.net · Sun 15 Nov

Sunday 15 November

MACaQ: Lower WikiText2 perplexity than MR-GPTQ and DuQuant++ on Llama-3-8B, faster quantization and a reported 64 minutes for 122B expert layers

devotd. — Clear, source-backed reporting on IT, AI, LLMs and local AI.

AI-generated summary Huawei CSL released MACaQ to create deployable 4-bit models by processing each linear layer independently to reduce quantization time and required GPU memory. In paper experiments on Llama-3-8B using WikiText2, MACaQ achieved lower perplexity than MR-GPTQ and the DuQuant++ family.