Activation Function
A mathematical formula introducing non-linear thresholds to model outputs, allowing networks to learn complex representations.
Activation Functions (like GeLU, ReLU, and Swish) introduce non-linear mapping inside neural network architectures. This allows nodes to learn complex patterns instead of simple linear relationships.
Historical figures and technical concepts for informational purposes only. Not technical, professional, legal, or financial advice. Sources: Official Documentation.