1
Towards Distributed Inference of LLMs on a P2P Network
新方案解决LLM推理中前缀缓存的跨节点共享难题，去中心化路由让集群推理更高效。
arXiv:2606.17059v1 Announce Type: cross Abstract: Prefix caching can reduce LLM inference latency by reusing KV caches across requests with shared pro…