1
Kara: Efficient Reasoning LLM Serving via Sliding-Window KV Cache Compression
通过滑动窗口KV缓存压缩，大幅降低大模型推理内存占用与延迟，提升服务吞吐量。
arXiv:2607.01237v1 Announce Type: cross Abstract: Reasoning language models often generate long chain-of-thought (CoT), which accumulates a massive KV…