AMD ROCm Blogs · Wed 11 Nov

AIAI-generated summary ROCm Blogs describes optimizing GLM-5.2 FP8 inference on eight AMD Instinct MI355X GPUs using end-to-end inference stack adjustments including EvalScope benchmarks. EvalScope measurements show peak throughput increasing from 163K to 629K tok/s at 64K and from 137K to 553K tok/s at 128K.

Long Context Inference Optimization Benchmarks

AIAI-generated summary GLM-5.3-Flash runs on an eight-GPU AMD Instinct MI300X node using a sparse mixture-of-experts architecture and a hybrid attention stack. The ROCm Blogs post details how tensor parallelism and data parallelism strategies impact long-context serving throughput and KV cache capacity.

Mixture of Experts Long Context Inference Optimization Model Serving