.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/03_model_selection/plot_01_crossvalidation_lvq.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_03_model_selection_plot_01_crossvalidation_lvq.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_03_model_selection_plot_01_crossvalidation_lvq.py:


================
Cross validation
================

In all previous examples we showed the training performance of the models. However,
in practice it is much more interesting how well a model performs on
unseen data, i.e., the generalizability of the model. We can use `crossvalidation`_ for this.

.. _crossvalidation: https://scikit-learn.org/stable/modules/cross_validation.html

.. GENERATED FROM PYTHON SOURCE LINES 12-26

.. code-block:: Python


    import numpy as np
    from sklearn.datasets import load_iris
    from sklearn.model_selection import (
        RepeatedKFold,
        cross_val_score,
    )
    from sklearn.pipeline import make_pipeline
    from sklearn.preprocessing import StandardScaler

    from sklvq import GMLVQ

    data, labels = load_iris(return_X_y=True)








.. GENERATED FROM PYTHON SOURCE LINES 27-30

Sklearn provides a very handy way of performing cross validation. For this
purpose we firstly create a pipeline and initiate a sklearn object that will
repeatedly create k folds for us.

.. GENERATED FROM PYTHON SOURCE LINES 30-58

.. code-block:: Python


    # Create a scaler instance
    scaler = StandardScaler()

    # Create a GMLVQ  model instance
    model = GMLVQ(
        distance_type="adaptive-squared-euclidean",
        activation_type="swish",
        activation_params={"beta": 2},
        solver_type="waypoint-gradient-descent",
        solver_params={"max_runs": 10, "k": 3, "step_size": np.array([0.1, 0.05])},
        random_state=1428,
    )

    # Link them together (Note this will work as it should in a CV setting, i.e., it's fitted to the
    # training data and predict is used for the testing data which makes sure the test data is
    # scaled using the tranformation parameters found during training.
    pipeline = make_pipeline(scaler, model)

    # Create an object that n_repeat times creates k=10  folds.
    repeated_10_fold = RepeatedKFold(n_splits=10, n_repeats=10)

    # Call the cross_val_score using all created instances and loaded data. Note it can accept
    # different and also multiple scoring parameters
    accuracy = cross_val_score(pipeline, data, labels, cv=repeated_10_fold, scoring="accuracy")

    # Print the mean and standard deviation of the cross validation testing scores.
    print(f"Accuracy, mean (std): {np.mean(accuracy):.2f} ({np.std(accuracy):.2f})")




.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Accuracy, mean (std): 0.97 (0.05)





.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 1.183 seconds)


.. _sphx_glr_download_auto_examples_03_model_selection_plot_01_crossvalidation_lvq.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_01_crossvalidation_lvq.ipynb <plot_01_crossvalidation_lvq.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_01_crossvalidation_lvq.py <plot_01_crossvalidation_lvq.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: plot_01_crossvalidation_lvq.zip <plot_01_crossvalidation_lvq.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_