A Mechanistic Study of Transformers Training Dynamics
揭秘Transformer训练中的动态机制,为AI可解释性提供新视角,大模型研究者必读。
arXiv:2410.24050v3 Announce Type: replace Abstract: Large-scale pretraining of transformers has been central to the success of foundation models. Howe…