Evolving Infra: Agent-Adaptive LLM Serving
AI-generated summary Researchers from MIT, UC Berkeley, and Georgia Tech submitted a preprint introducing an open-source inference system called Evolving Infra that dynamically adapts compute allocation and key-value cache management to multi-agent workloads. The authors evaluated Evolving Infra on NVIDIA GB300 GPUs serving GLM-5.2 and Kimi K2.6 models across coding tasks, theorem proving, and multimodal UI navigation workloads.