1
PowLU: An Activation Function for Stable Pre-Training of LLMs
提出PowLU激活函数新方案，专门解决大语言模型预训练过程的不稳定性问题。
arXiv:2605.25704v1 Announce Type: cross Abstract: In contemporary large language models (LLMs), the swish-gated linear unit (SwiGLU) activation functi…