Ranking Before Serving: Low-Latency LLM Serving via Pairwise Learning-to-Rank
利用成对学习排序技术,在推理前优化请求排序,大幅降低大模型服务延迟。
arXiv:2510.03243v3 Announce Type: replace-cross Abstract: Efficient scheduling of large language model (LLM) inference tasks is critical for achieving…