1
LLM Serving Optimization with Variable Prefill and Decode Lengths
固定KV缓存内存下，智能调度不同长度请求，突破LLM推理效率瓶颈。
arXiv:2508.06133v4 Announce Type: replace-cross Abstract: We study offline scheduling for large language model (LLM) serving under a fixed KV-cache me…