1
Service-Induced Congestion in Memory-Constrained LLM Serving
揭示LLM服务中内存受限导致的服务诱导拥塞现象，为推理系统优化提供新视角。
arXiv:2606.15555v1 Announce Type: cross Abstract: In large language model (LLM) serving, each request accumulates persistent graphics processing unit …