LLM Quantization: What fp8 and fp4 Do to Quality and Price
openrouter.ai — When you call an open-weight model through an API, several providers may serve it at different precisions, such as 16-bit bf16, 8-bit fp8 or 4-bit fp4. The
Quantization
I stopped chasing bigger local models, and a 27B model on my 8GB GPU does more of what I need
www.xda-developers.com — I don't download every new model anymore
My 12GB RTX 4070 Ti runs multiple DLSS 5 layers thanks to one OptiScaler setting
XDA — Nothing like an open-source tool to squeeze more power
My gaming GPU sat idle most of the day, so I turned it into a local AI server the whole house can use
XDA — One Llama.cpp command can turn your PC into an AI server, and it really is that easy
Online Quantization: Smooth Path from One Checkpoint to Any Desired Serving Scenario
ROCm Blogs — Learn how AMD Quark's online quantization quantizes LLM weights at load time across vLLM, ATOM, and SGLang, with per-layer mixed FP8/MXFP4 recipes.
Five Ways NVIDIA Co-Design Maximizes the Value of Every Byte in the AI Factory
NVIDIA Technical Blog — In an AI factory, high bandwidth memory (HBM)is the fastest, most expensive, and most contested memory tier. Every byte spent on a redundant weight copy, an idle session’s KV cache…
Quark Mixed-Precision Quantization for Wan2.2 Video Generation on AMD Instinct GPUs
ROCm Blogs — Apply AMD Quark mixed-precision MXFP4 quantization to Wan2.2-T2V-A14B video generation with calibration-driven block selection.
MiniMax M3.1 Flash, on more of your hardware.
Local Inference with Large Language Models
Dismiss email signup
Subscribe to 
The top AI news in your email delivered each morning. Choose categories