1
MemDecay: Region-Aware KV Cache Eviction for Efficient LLM Agent Inference
新方法MemDecay基于区域感知的KV缓存淘汰,大幅提升LLM Agent推理效率,降低内存占用。
arXiv:2607.10582v1 Announce Type: cross Abstract: Large language model (LLM) agents accumulate heterogeneous context, including system instructions, p…