1
CONF-KV: Confidence-Aware KV Cache Eviction with Mixed-Precision Storage for Long-Horizon LLM
利用解码步骤的置信度信号，提出混合精度存储的KV缓存逐出策略，有效缓解长序列推理中的GPU内存瓶颈。
arXiv:2605.24786v1 Announce Type: cross Abstract: Long-horizon LLM inference turns the key--value (KV) cache into the dominant GPU memory consumer and…