Activation Functions — Interactive Explorer
Plot, differentiate, and annotate every major activation function — adjust parameters live
© 2026 Greg T. Chism ·
MIT License
1
ReLU Family
2
Sigmoid & Tanh
3
Maxout Units
4
Linear Units
★
Compare All
Function
ReLU
Leaky ReLU
Abs Value
ELU
Swish
α
0.010
negative-slope coefficient
α
1.00
saturation level for x<0
β
1.00
sharpness / temperature
Equation
f(x) = max(0, x)
Properties
f(x) — activation
f′(x) — derivative
f(x)
f′(x)
identity / zero reference
Function
Sigmoid
Tanh
β
1.00
temperature (sharpness of transition)
Equation
f(x) = 1 / (1 + e^(−βx))
Properties
f(x) — activation
f′(x) — derivative
f(x)
f′(x)
saturation bounds
Number of linear pieces k
k = 2
k = 3
k = 4
k = 5
Maxout takes the max over k learned linear functions. Adjust each affine piece's slope and intercept below.
Equation
h(x) = max
j∈[k]
(w
j
x + b
j
)
Properties
h(x) — maxout output
h′(x) — derivative (a.e.)
Parameters
w
1.00
slope (weight)
b
0.00
bias (intercept)
Equation
f(x) = wx + b
Properties
f(x) — activation
f′(x) — derivative
f(x)
f′(x) = w (constant)
All Activation Functions — Function Plot (default parameters)