1
Beyond Prediction: Tail-Aware Scheduling for LLM Inference
大模型推理的延迟"长尾"痛点有解了！本文提出GPU调度新方法UniBoost，通过感知输出长度分布，显著降低高响应百分位延迟。
Article URL: https://yl3469.github.io/uniboost-icml26/ Comments URL: https://news.ycombinator.com/item?id=48808378 Points: 2 # Comments: 0