Vegas: Self-Speculative Decoding with Verification-Guided Sparse Attention
用验证引导的稀疏注意力加速长上下文LLM推理,自推测解码摆脱KV缓存瓶颈。
arXiv:2602.07223v2 Announce Type: replace Abstract: Long-context large language model (LLM) inference has become the norm for today's AI applications.…