面向长推理场景，提出信息感知的KV缓存压缩方法，显著提升大模型推理效率与速度。
arXiv:2606.26875v1 Announce Type: cross Abstract: Reasoning capability has advanced rapidly in large language models (LLMs), leading to an increasing …
扩散模型推理还能更省？预算约束下的步骤级缓存策略，为生成式AI降本增效提供新思路。
arXiv:2606.13496v1 Announce Type: new Abstract: Step-level caching accelerates diffusion models by exploiting temporal redundancy across denoising ste…
Python 库 Llmbuffer 通过分离不可变前缀与可变尾部，实现 LLM 对话历史的缓存优化，大幅减少重复计算。
I was not getting good cache utilization when including dynamic context in agent threads. After a lot of experimentation, I found a good pattern that …
实测LLM提示缓存：相同提示从0%到91%命中率，揭示不同提供商缓存策略的巨大差异。
We run an AI companion bot. Every chat turn, the model sees the same ~5K-token prefix — character persona, content-tier rules, formatting guardrails, …