{ "cells": [ { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "%matplotlib inline" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n# Local Generalized Matrix LVQ (LGMLVQ)\n\nExample of how to use LGMLVQ `[1]`_ on the classic iris dataset.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "import matplotlib\nimport matplotlib.pyplot as plt\nimport numpy as np\nfrom sklearn.datasets import load_iris\nfrom sklearn.metrics import classification_report\nfrom sklearn.preprocessing import StandardScaler\n\nfrom sklvq import LGMLVQ\n\nmatplotlib.rc(\"xtick\", labelsize=\"small\")\nmatplotlib.rc(\"ytick\", labelsize=\"small\")\n\n# Contains also the target_names and feature_names, which we will use for the plots.\niris = load_iris()\n\ndata = iris.data\nlabels = iris.target" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Fitting the Model\nScale the data and create a LGMLVQ object with, e.g., custom distance function, activation\nfunction and solver. See the API reference under documentation for defaults and other\npossible parameters.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "# Sklearn's standardscaler to perform z-transform\nscaler = StandardScaler()\n\n# Compute (fit) and apply (transform) z-transform\ndata = scaler.fit_transform(data)\n\n# The creation of the model object used to fit the data to.\nmodel = LGMLVQ(\n relevance_localization=\"class\", # Can either be \"class\" or \"prototypes\"\n distance_type=\"local-adaptive-squared-euclidean\",\n activation_type=\"swish\",\n activation_params={\"beta\": 2},\n solver_type=\"lbfgs\",\n)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The next step is to fit the LGMLVQ object to the data and use the predict method to make the\npredictions. Note that this example only works on the training data and therefor does not say\nanything about the generalizability of the fitted model.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "# Train the model using the scaled data and true labels\nmodel.fit(data, labels)\n\n# Predict the labels using the trained model\npredicted_labels = model.predict(data)\n\n# To get a sense of the training performance we could print the classification report.\nprint(classification_report(labels, predicted_labels))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Extracting the Relevance Matrices\nIn addition to the prototypes (see GLVQ example), LGMLVQ learns a number of matrices `lambda_`\nwhich can tell us something about which features are most relevant for the classification per\nclass. The number of relevance matrices is determined by the number of prototypes used per\nclass as well as which localization strategy is used. It can either be a relevance matrix per\nclass (even if there are more prototypes for that class they will share the relevance matrix.\nOr a relevance matrix per prototype, where each prototype (even if they have the same class)\nhas its own matrix.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "colors = [\"blue\", \"red\", \"green\"]\nnum_prototypes = model.prototypes_.shape[0]\nnum_features = model.prototypes_.shape[1]\n\nfig, ax = plt.subplots(num_prototypes, 1)\nfig.suptitle(\"Relevance Diagonal of each Prototype's lambda matrix\")\n\nfor i, lambda_ in enumerate(model.lambda_):\n ax[i].bar(\n range(num_features),\n np.diagonal(lambda_),\n color=colors[i],\n label=iris.target_names[model.prototypes_labels_[i]],\n )\n ax[i].set_xticks(range(num_features))\n if i == (num_prototypes - 1):\n ax[i].set_xticklabels([name[:-5] for name in iris.feature_names])\n else:\n ax[i].set_xticklabels([], visible=False)\n ax[i].tick_params(\n axis=\"x\", which=\"both\", bottom=False, top=False, labelbottom=False\n )\n ax[i].set_ylabel(\"Weight\")\n ax[i].legend()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that each diagonal still adds up to one (See GMLVQ example). However, each diagonal\nsummarizes the importance of the features for its corresponding class versus all other classes.\n\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Transforming the Data\nIn addition to making predictions LGMLVQ can be used to transform the data using the\neigenvectors of the relevance matrices. In contrast to GMLVQ this can be done for each of the\nmatrices separately.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "# This will return a 3D shape with the 1st and 2nd axes representing the data (n_observations,\n# n_eigenvectors). The third axis are the different relevance matrices\n\nt_d = model.transform(data, omega_hat_index=[0, 1, 2])[:, :2, :]\nt_d = np.transpose(t_d, axes=(2, 0, 1)) # shape that is easier to work with\n\nt_m = model.transform(model.prototypes_, omega_hat_index=[0, 1, 2])[:, :2, :]\nt_m = np.transpose(t_m, axes=(2, 0, 1)) # shape that is easier to work with\n\nfig, ax = plt.subplots(num_prototypes, 1, figsize=(6.4, 14.4))\nfig.tight_layout(pad=6.0)\n\ncolors = [\"blue\", \"red\", \"green\"]\nfor i, xy_dm in enumerate(zip(t_d, t_m)):\n xy_d = xy_dm[0]\n xy_m = xy_dm[1]\n for j, cls in enumerate(model.classes_):\n ii = cls == labels\n ax[i].scatter(\n xy_d[ii, 0],\n xy_d[ii, 1],\n c=colors[j],\n s=100,\n alpha=0.7,\n edgecolors=\"white\",\n label=iris.target_names[model.prototypes_labels_[j]],\n )\n ax[i].scatter(\n xy_m[:, 0],\n xy_m[:, 1],\n c=colors,\n s=180,\n alpha=0.8,\n edgecolors=\"black\",\n linewidth=2.0,\n )\n ax[i].title.set_text(\n \"Relevance projection w.r.t. {}\".format(\n iris.target_names[model.prototypes_labels_[i]]\n )\n )\n ax[i].set_xlabel(\"First eigenvector\")\n ax[i].set_ylabel(\"Second eigenvector\")\n ax[i].legend()\n ax[i].grid(True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## References\n_`[1]` Schneider, P., Biehl, M., & Hammer, B. (2009). \"Adaptive Relevance Matrices in Learning\nVector Quantization\" Neural Computation, 21(12), 3532\u20133561, 2009.\n\n" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.9" } }, "nbformat": 4, "nbformat_minor": 0 }