{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "%matplotlib inline"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "\n# Activation Functions\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "from typing import Union\n\nimport numpy as np\nfrom sklearn.datasets import load_iris\nfrom sklearn.metrics import classification_report\n\nfrom sklvq import GLVQ\nfrom sklvq.activations import ActivationBaseClass\n\ndata, labels = load_iris(return_X_y=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "The sklvq contains already a few activation function. Please see the API reference\nunder Documentation. However, it is fairly easy to create your own. The package\nworks with callable classes and provides a base class for convenience. The base class\nfor the activation functions is sklvq.activations.ActivationBaseClass` and does\nnothing more then tell you to implement a `__call__()` and `gradient()` method.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "# This is the implementation of sklvq.activations.Sigmoid with some additional comments\nclass CustomSigmoid(ActivationBaseClass):\n\n    # Activation callables can have a custom init of which the parameters can be passed\n    # through the `activation_params (Dict)' parameter of the LVQ algorithms. Or the\n    # object can just be initialized before hand.\n    def __init__(self, beta: Union[int, float] = 1):\n        self.beta = beta\n\n    # The activation call function needs to apply the activation elementwise on x.\n    def __call__(self, x: np.ndarray) -> np.ndarray:\n        return np.asarray(1 / (np.exp(-self.beta * x) + 1))\n\n    # The gradient is the elementwise derivative of the activation function.\n    def gradient(self, x: np.ndarray) -> np.ndarray:\n        exp = np.exp(self.beta * x)\n        return np.asarray((self.beta * exp) / (exp + 1) ** 2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "The CustomSigmoid above, accompanied with some tests and documentation, would make a\ngreat addition to the sklvq package. However, it can also directly be passed to\nthe algorithm.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "model = GLVQ(activation_type=CustomSigmoid, activation_params={\"beta\": 2})\n\nmodel.fit(data, labels)\n\n# Predict the labels using the trained model\npredicted_labels = model.predict(data)\n\n# Print a classification report (sklearn)\nprint(classification_report(labels, predicted_labels))"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.9"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}