1
FORGE: Fused On-Register Gradient Elimination for Memory-Efficient LLM Training
融合反向传播与优化器阶段的梯度处理，大幅降低LLM训练的内存峰值，突破显存瓶颈的新方案。
arXiv:2606.22932v1 Announce Type: new Abstract: Reverse-mode differentiation computes every weight gradient, writes it to memory, and only then lets t…