Evolving Infra: Agent-Adaptive LLM Serving at Scale
Agentic workloads are a major source of LLM inference demand. Existing prefill--decode (PD) disaggregated serving systems offer limited flexibility for runtime adaptation of compute allocation and KV
- Upvotes12