sklvq.activations.Swish

class sklvq.activations.Swish(beta: Union[int, float] = 1)[source]

Swish function

Class that holds the swish function and gradient as discussed in [1]

Parameters
betaint, float, default=1

Positive non-zero value that controls the steepness of the Swish function.

See also

Identity, Sigmoid, SoftPlus

References

[1] Villmann, T., Ravichandran, J., Villmann, A., Nebel, D., & Kaden, M. (2019). “Activation Functions for Generalized Learning Vector Quantization - A Performance Comparison”, 2019.

__call__(x: numpy.ndarray) → numpy.ndarray[source]
Implements the swish function:

f(\mathbf{x}) = \frac{\mathbf{x}}{1 + e^{-\beta \cdot \mathbf{x}}}

Parameters
xndarray of any shape
Returns
ndarray of shape (x.shape)

Elementwise evaluation of the swish function.

gradient(x: numpy.ndarray) → numpy.ndarray[source]
Implements the sigmoid function’s gradient:

\frac{\partial f}{\partial \mathbf{x}} = \beta \cdot f(\mathbf{x}) + (\frac{1}{1 + e^{-\beta \cdot \mathbf{x}}}) \cdot (1 - \beta \cdot f(\mathbf{x}))

Parameters
xndarray of any shape
Returns
ndarray of shape (x.shape)

Elementwise evaluation of the swish function’s gradient.