AdamW Optimizer
A variant of the Adam optimizer that decouples weight decay from gradient updates to improve regularization.
AdamW is the default optimizer for training transformers. By separating weight decay from standard gradient calculations, it improves model generalization compared to standard Adam.
Historical figures and technical concepts for informational purposes only. Not technical, professional, legal, or financial advice. Sources: Official Documentation.