块稀疏注意力硬件友好却未被主流LLM采用？本文剖析原因并提出新型COBS注意力机制。
arXiv:2607.09052v1 Announce Type: new Abstract: Block sparse attention is a hardware friendly way to alleviate the key-value (KV) cache read bottlenec…
提出预测-重用-修复机制，动态稀疏注意力加速长上下文LLM解码，降低推理延迟。
arXiv:2606.30389v1 Announce Type: new Abstract: Dynamic sparse attention (DSA) accelerates long-context LLM decoding by attending to only the top-K KV…
提出一种新型稀疏块注意力机制Proxy Block-CAGE，有望提升长序列建模效率并降低计算开销。
Hi, I'm a PhD student in Bioinformatics/Computational Biology with a software engineering background, I'm trying to pivot toward AI/ML research. I'm f…
视频扩散模型新突破：细粒度稀疏注意力FG-Attn，大幅提升计算效率与生成质量。
arXiv:2509.16518v2 Announce Type: replace Abstract: Using diffusion transformers for media generation may require evaluating attention over extremely …
针对AI代理的高效可编程稀疏注意力服务新框架，大幅降低计算成本并保持灵活性。
arXiv:2606.06453v1 Announce Type: new Abstract: Sparse attention is becoming increasingly important for serving large language models (LLMs) as genera…
腾讯混元提出Stem稀疏注意力算法，从因果信息流革新块稀疏，首字延迟降低3.6倍，已被ICML-26收录。
IT之家 6 月 5 日消息，腾讯混元今日宣布提出 Stem 稀疏注意力算法，已被机器学习顶会 ICML-26 收录。 官方表示，Stem 稀疏注意力算法从“因果信息流”重新审视块级稀疏，用 Token 位置衰减（TPD）和输出感知度量（OAM）两大创新， 仅用 25% 算力就逼近稠密注意力的精度 …
跨层共享路由实现稀疏注意力，一次索引大幅降低计算开销，为长序列模型提供高效新范式。
arXiv:2606.06467v1 Announce Type: cross Abstract: Long-context inference in modern LLMs is increasingly constrained by decoding efficiency, especially…
稀疏注意力新突破：解耦选择与计算，解决KV缓存瓶颈与PCIe传输困境。
arXiv:2606.04511v1 Announce Type: new Abstract: Sparse attention reduces compute and memory bandwidth for long-context LLM inference. However, two key…
用验证引导的稀疏注意力加速长上下文LLM推理，自推测解码摆脱KV缓存瓶颈。
arXiv:2602.07223v2 Announce Type: replace Abstract: Long-context large language model (LLM) inference has become the norm for today's AI applications.…
LLM长上下文推理新突破：无需额外训练即可精准估计KV缓存重要性，大幅降低推理开销。
arXiv:2605.27740v1 Announce Type: new Abstract: Long-context inference in large language models (LLMs) is bottlenecked by the linear growth of the sel…
小米 MiMo API 以极致稀疏比实现低成本推理，永久降价高达 99%，兼顾 AI 对话、语音识别与合成，是性价比超高的模型服务。
IT之家 5 月 28 日消息，小米 MiMo 官方昨日宣布， 小米 MiMo-V2.5 系列 API 永久降价 ，相比原始 API 定价，新定价最高降幅可达 99%，且不再区分上下文窗口长度。 小米 MiMo 负责人罗福莉随后在 X 平台发文，谈到了 MiMo API 降价的技术原因，IT之家附罗…
光谱感知的块稀疏注意力机制，突破长上下文LLM预填充效率瓶颈
arXiv:2602.08426v2 Announce Type: replace-cross Abstract: Block-sparse attention is promising for accelerating long-context LLM pre-filling, yet ident…
探讨推理时上下文稀疏性的真实价值，是幻觉陷阱还是效率提升突破口？
arXiv:2605.24168v1 Announce Type: new Abstract: Sparsity has long been a central theme in LLM efficiency, but its role in context processing remains u…
新方法MTraining通过分布式动态稀疏注意力，大幅降低超长上下文训练的计算开销。
arXiv:2510.18830v2 Announce Type: replace-cross Abstract: The adoption of long context windows has become a standard feature in Large Language Models …
提出可微分自适应稀疏层次注意力机制，显著提升长序列建模效率与计算可扩展性
arXiv:2605.18753v1 Announce Type: cross Abstract: Current hierarchical attention methods, such as NSA and InfLLMv2, select the top-k relevant key-valu…
突破大模型长上下文推理瓶颈，百步内将全注意力高效转为稀疏，平衡效率与精度。
arXiv:2605.16928v1 Announce Type: new Abstract: Long-context inference in large language models is bottlenecked by the quadratic cost of full attentio…
提出无需重训练的稀疏注意力机制STS，通过推测性token稀疏性突破大模型长序列推理的算力和内存瓶颈。
arXiv:2605.15508v1 Announce Type: new Abstract: The quadratic complexity of attention imposes severe memory and computational bottlenecks on Large Lan…