AMD ROCm Blogs

AIAI-generated summary ROCm Blogs describes optimizing GLM-5.2 FP8 inference on eight AMD Instinct MI355X GPUs using end-to-end inference stack adjustments including EvalScope benchmarks. EvalScope measurements show peak throughput increasing from 163K to 629K tok/s at 64K and from 137K to 553K tok/s at 128K.

Long Context Inference Optimization Benchmarks

AIAI-generated summary GLM-5.3-Flash runs on an eight-GPU AMD Instinct MI300X node using a sparse mixture-of-experts architecture and a hybrid attention stack. The ROCm Blogs post details how tensor parallelism and data parallelism strategies impact long-context serving throughput and KV cache capacity.

Mixture of Experts Long Context Inference Optimization Model Serving

AIAI-generated summary ROCm Blogs reports the release of MONAILabel 0.8.5 as an Early Access release for AMD Instinct GPUs, bringing interactive medical-image annotation support. ROCm Blogs says the implementation enables model inference to run up to 131 times faster than on a host CPU during spleen CT segmentation workloads.

GPUs Inference Optimization Healthcare

AIAI-generated summary ROCm Blogs presents InferaSim to replace most trial measurements with fast projection when configuring large language model serving. ROCm Blogs says the tool evaluates thousands of candidate configurations using fewer GPU hours and helps identify optimal serving setups.

Inference Optimization Model Serving Compute Costs

AIAI-generated summary ROCm Blogs presents a guide detailing how to profile a DeepSeek-V4-Flash training proxy in Primus using madengine and rocprofv3. The workload uses an eight-layer BF16 training proxy configured with eight AMD Instinct MI350X GPUs on a single node.

GPUs Distributed Training Inference Optimization Kernels

AIAI-generated summary ROCm Blogs introduces online quantization, a new feature that runs during model loading inside the serving engine to bypass separate export scripts and disk writes. ROCm Blogs says the approach suits scenarios like reinforcement training rollouts, format requantization, mixed-precision exploration, and minor checkpoint modifications.

Inference Optimization Model Serving Quantization

AIAI-generated summary AMD Quark uses mixed-precision quantization to evaluate transformer blocks for video generation on AMD Instinct GPUs. The calibration policy promotes sensitive blocks to W4A8 while keeping others on the W4A4 path.

GPUs Inference Optimization Quantization Video Generation

AIAI-generated summary FlyDSL allows developers to write GPU programs in Python and compile specialized kernels on demand for LLM serving. ATOM integrates FlyDSL mono decode for models including MiniMax-M3, GLM-5.2, Kimi-K3, and DeepSeek-V4.1.

Inference Optimization Model Serving Kernels

AIAI-generated summary ROCm Blogs has made available GLM-5.2 AgentX on AMD Instinct MI355X to boost throughput and interactivity. ROCm Blogs states that cache efficiency, faster decode kernels, careful AgentX validation, and upstream collaboration raised peak throughput 1.72 times and peak P90 interactivity 2.46 times.

GPUs Inference Optimization Kernels

AIAI-generated summary AMD released a guide and container configurations utilizing xDiT to scale diffusion inference across one, two, or four Radeon AI PRO R9700S GPUs. AMD states that using multiple GPUs can accommodate a model exceeding 32GB of memory and reduce generation time when configured to limit PCIe communication.

Inference Optimization Image Generation Video Generation

AIAI-generated summary ROCm Blogs has made available a method to accelerate Stable Diffusion image generation on AMD Ryzen AI neural processing units using TeaCache. ROCm Blogs states that this approach reuses intermediate outputs across the denoising loop to skip redundant computation without requiring model retraining.

Inference Optimization Edge AI Image Generation

AIAI-generated summary AMD is releasing an early access update that brings validated and optimized inference support for five Medical Open Network for AI Model Zoo bundles on AMD Instinct GPUs. ROCm Blogs says the five bundles clear upstream Sørensen-Dice overlap score thresholds with no changes to model weights and no changes to inference logic.

GPUs Inference Optimization AI for Science