Inference optimization encompasses the techniques and software used to reduce the computational cost and latency of running artificial intelligence models in production. Truncated covers hardware acceleration methods, quantization strategies, compiler frameworks and performance benchmarks.
NVIDIA Technical Blog — During reinforcement learning (RL) post-training, training trajectories are generated on the fly by running inference with the current policy model. For training efficiency, it is common to run…
GamesBeat — Hollywood & Games returns to Los Angeles on December 10, bringing together 250+ executives, founders, investors and decision-makers at the Grammy Museum's Clive Davis...
OpenRouter Blog — OpenRouter's next generation Auto Router and the new Auto Code router pick a model and reasoning effort for each request from the model pool your guardrails, ZDR, provider restrictions, and BYOK policies allow. Auto Router for general agents and apps, Auto Code for coding agents and harnesses.
ROCm Blogs — Learn how ROCm InferaSim predicts LLM serving performance on AMD Instinct GPUs from a few short runs, so you can deploy the best configuration.
Crunchbase News — AI chip startups can challenge the sector's Goliath's by building substantially better hardware for emerging inference demands, much as Juniper challenged Cisco with a new architecture suited to the internet’s rapid growth. To win customers, writes guest author R K Anand, co-founder and chief product officer of Tensordyne, challengers must prove their advantages, make switching easy and use small, agile teams to anticipate needs that incumbents have yet to address.
Interesting Engineering — VFlash achieved 70.2% average robot manipulation success, compared with 62.6% for the baseline.
ROCm Blogs — Profile an eight-layer training proxy derived from the DeepSeek-V4-Flash Primus configuration on eight MI350X GPUs with madengine and TraceLens, then turn operator-to-kernel insights into faster runs.
ROCm Blogs — Learn how AMD Quark's online quantization quantizes LLM weights at load time across vLLM, ATOM, and SGLang, with per-layer mixed FP8/MXFP4 recipes.
NVIDIA Technical Blog — In an AI factory, high bandwidth memory (HBM)is the fastest, most expensive, and most contested memory tier. Every byte spent on a redundant weight copy, an idle session’s KV cache…
octomind — Flash Attention explained: what VRAM it saves, what stays in the KV cache, and exact Ollama, llama.cpp, vLLM and PyTorch settings, checked October 2026.
ROCm Blogs — Apply AMD Quark mixed-precision MXFP4 quantization to Wan2.2-T2V-A14B video generation with calibration-driven block selection.
AMD — Discover how Kimi K3.1 Pro runs on AMD Instinct™ MI355X GPUs from day 0 with ATOM, ROCm™ software, and optimized inference kernels.
AMD — Blackfuel creates first AMD-powered inference deployment for more open, outcome-driven future for AI infrastructure
Engadget — The NVIDIA modding platform lets you remaster classic games with modern graphics, and now DLSS 5 is included.
ROCm Blogs — FlyDSL mono kernels cut decode latency in ATOM, with MiniMax-M3 and DeepSeek-V4.1 results.
Engineering at Meta — Facebook Feature Framework (F3) is Meta’s framework for authoring, compiling, and serving the machine learning features underlying ads ranking models, where a single request can require more …
Tech Xplore — Researchers have developed technologies that reduce communication overhead by transmitting only the data needed for artificial intelligence (AI) analysis and shorten the time users must wait for a response when returning to an AI application after using other apps on a smartphone.
Fortune — Running AI now costs more than training it.
Wccftech — Astral Chain's unofficial native PC port reaches remaster territory thanks to NVIDIA DLSS 5 neural rendering tests
The top AI news in your email delivered each morning. Choose categories