1
Show HN: KVBoost – chunk-level KV cache reuse for HuggingFace, 5–48x faster TTFT
KV缓存块级复用，HuggingFace推理首Token速度提升5-48倍，轻松接入现有模型。
Article URL: https://pythongiant.github.io/KVBoost/ Comments URL: https://news.ycombinator.com/item?id=48232060 Points: 18 # Comments: 12