1
Pipeline-parallel LLM inference across GPUs on separate machines
744B超大规模模型通过分布式流水线并行在多个消费级GPU上高效推理，速度达30 tok/s，展示前沿部署方案。
Article URL: https://github.com/leyten/shard Comments URL: https://news.ycombinator.com/item?id=48602121 Points: 1 # Comments: 0