1
PEEK: Predictive Queue-Informed KV Cache Management for LLM Serving
预测性队列感知KV缓存管理方案，让LLM服务缓存命中率与吞吐量提升数倍，超越SGLang/vLLM基线。
arXiv:2607.02525v1 Announce Type: cross Abstract: We present PEEK, a lightweight scheduling and eviction framework for both online (streaming) and off…