块稀疏注意力硬件友好却未被主流LLM采用？本文剖析原因并提出新型COBS注意力机制。
arXiv:2607.09052v1 Announce Type: new Abstract: Block sparse attention is a hardware friendly way to alleviate the key-value (KV) cache read bottlenec…
预测性队列感知KV缓存管理方案，让LLM服务缓存命中率与吞吐量提升数倍，超越SGLang/vLLM基线。
arXiv:2607.02525v1 Announce Type: cross Abstract: We present PEEK, a lightweight scheduling and eviction framework for both online (streaming) and off…
通过滑动窗口KV缓存压缩，大幅降低大模型推理内存占用与延迟，提升服务吞吐量。
arXiv:2607.01237v1 Announce Type: cross Abstract: Reasoning language models often generate long chain-of-thought (CoT), which accumulates a massive KV…
面向设备端LLM推理的KV缓存碎片引擎，跳过最昂贵推理环节，专为ARM64 Android优化。
Article URL: https://github.com/bossandboss/EdgeSync-LLM Comments URL: https://news.ycombinator.com/item?id=48732973 Points: 2 # Comments: 0
新论文提出ReFreeKV，无需预设阈值的KV缓存压缩，实现无损内存缩减，解决现有方法依赖人工设定预算的痛点。
arXiv:2502.16886v4 Announce Type: replace-cross Abstract: To reduce memory consumption during LLM inference, a handful of methods have been proposed f…
利用Linux系统压力信息动态修剪大模型KV缓存，专为边缘设备优化内存使用
I thought it'd be interesting to use Linux PSI (Pressure Stall Information) for an LLM runtime to trim the KV cache. This is mainly useful imo for edg…
新方法GELO专为共享加速器场景设计，有效保护开源大模型的提示词隐私，对抗内存读取攻击。
arXiv:2603.05035v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are increasingly served on shared accelerators where an adversa…
多步LLM推理新突破：通过推理感知KV缓存共享与自信提前退出机制，大幅提升效率，已被ICML 2026 Workshop收录。
arXiv:2606.09937v1 Announce Type: cross Abstract: We introduce RKSC (Reasoning-Aware KV Cache Sharing), a training-free inference framework that elimi…
LLM Wiki静态缓存假设不再适用，这篇论文提出了主动物性评分钉住方案，解决时变知识流式编译难题。
arXiv:2606.09877v1 Announce Type: new Abstract: LLM wiki systems compile knowledge into pre-filled KV caches for efficient inference, but assume a sta…
针对Agent推理场景的跨轮意图感知KV缓存剪枝方法，显著降低内存占用与延迟。
arXiv:2606.09916v1 Announce Type: cross Abstract: Multi-turn LLM agents fan short queries into long trajectories of tool calls, search results, and in…
扩散语言模型推理加速新方案，通过共享前缀KV缓存显著提升效率，结构简洁效果亮眼。
arXiv:2606.07571v1 Announce Type: new Abstract: Key-value (KV) caching for shared prefixes is essential for high-throughput large language model (LLM)…
稀疏注意力新突破：解耦选择与计算，解决KV缓存瓶颈与PCIe传输困境。
arXiv:2606.04511v1 Announce Type: new Abstract: Sparse attention reduces compute and memory bandwidth for long-context LLM inference. However, two key…
自适应多示例提示学习，用语义感知缓存复用提升LLM推理效率
arXiv:2605.03644v2 Announce Type: replace Abstract: Many-Shot In-Context Learning (ICL) has emerged as a promising paradigm, leveraging extensive exam…
用验证引导的稀疏注意力加速长上下文LLM推理，自推测解码摆脱KV缓存瓶颈。
arXiv:2602.07223v2 Announce Type: replace Abstract: Long-context large language model (LLM) inference has become the norm for today's AI applications.…
提出多LoRA LLM代理间KV缓存共享机制，有效降低推理计算开销，为服务优化提供新思路。
arXiv:2602.01053v2 Announce Type: replace Abstract: Role specialization in multi-LLM agent systems is often realized via multi-LoRA, where agents shar…
小波引导KV缓存过滤，低成本扩展扩散大模型处理超长上下文的能力
arXiv:2606.00724v1 Announce Type: cross Abstract: Diffusion Large Language Models (DLMs) have demonstrated significant advantages across various tasks…
突破传统KV缓存局限，专门针对Agentic LLM的推理优化，引入策略驱动编辑的新范式。
arXiv:2606.01065v1 Announce Type: cross Abstract: Modern KV cache management assumes the chatbot workload: prompts arrive once and the cache grows app…
无需训练的全局回归方法，高效压缩长上下文LLM的KV缓存，兼顾速度与精度。
arXiv:2605.31105v1 Announce Type: new Abstract: Large language models (LLMs) with extended context lengths rely on the key-value (KV) cache to support…
提出 OBCache 最优脑剪枝方法，精准削减 KV 缓存，让长上下文大模型推理又快又省内存。
arXiv:2510.07651v2 Announce Type: replace-cross Abstract: Large language models (LLMs) with extended context windows enable powerful applications but …
LLM长上下文推理新突破：无需额外训练即可精准估计KV缓存重要性，大幅降低推理开销。
arXiv:2605.27740v1 Announce Type: new Abstract: Long-context inference in large language models (LLMs) is bottlenecked by the linear growth of the sel…