llm-d 1.0 targets production inference bottlenecks
AI-generated summary Contributors have released llm-d 1.0 to add multi-tenant quality of service, zero-downtime operations, a stable configuration API and agent-aware serving. IBM Research reported 11.5 times lower inference cost than an all-proprietary set-up for work exploring quantum error-correction hardware, with a prefix-cache hit rate of about 98 percent.