.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/05_customization/plot_04_solvers.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_05_customization_plot_04_solvers.py: ======= Solvers ======= .. GENERATED FROM PYTHON SOURCE LINES 6-24 .. code-block:: Python from typing import TYPE_CHECKING import numpy as np from sklearn.datasets import load_iris from sklearn.metrics import classification_report from sklearn.utils import shuffle from sklvq import GLVQ from sklvq.objectives import ObjectiveBaseClass from sklvq.solvers import SolverBaseClass from sklvq.solvers._base import _update_state if TYPE_CHECKING: from sklvq.models import LVQBaseClass STATE_KEYS = ["variables", "nit", "fun", "step_size"] .. GENERATED FROM PYTHON SOURCE LINES 25-27 The sklvq package contains a number of different solvers. Please see the API reference under Documentation for the full list. .. GENERATED FROM PYTHON SOURCE LINES 27-124 .. code-block:: Python class CustomSteepestGradientDescent(SolverBaseClass): def __init__( self, # init requires the objective instance to be given when initialized. It will be passed # to the (super) solver base class. objective: ObjectiveBaseClass, max_runs: int = 10, batch_size: int = 1, step_size: float = 0.1, callback: callable = None, ): super().__init__(objective) # In the actual implementation checks can be done to ensure proper values for the # parameters of the solver (as is done in the actual code). self.max_runs = max_runs self.batch_size = batch_size self.step_size = step_size self.callback = callback def solve( self, data: np.ndarray, labels: np.ndarray, model: "LVQBaseClass", ): # Calls the callback function is provided with the initial values. if self.callback is not None: state = _update_state( STATE_KEYS, variables=np.copy(model.get_variables()), nit="Initial", fun=self.objective(model, data, labels), ) if self.callback(state): return batch_size = self.batch_size # These checks cannot be done in init because data is not available at that moment. if batch_size > data.shape[0]: raise ValueError("Provided batch_size is invalid.") if batch_size <= 0: batch_size = data.shape[0] for i_run in range(self.max_runs): # Randomize order of samples shuffled_indices = shuffle(np.array(range(labels.size)), random_state=model.random_state_) # Divide the shuffled indices into batches (not necessarily equal size, # see documentation of numpy.array_split). batches = np.array_split( shuffled_indices, list(range(batch_size, labels.size, batch_size)), axis=0, ) # Update step size using a simple annealing strategy step_size = self.step_size / (1 + i_run / self.max_runs) for i_batch in batches: # Select the data batch = data[i_batch, :] batch_labels = labels[i_batch] # Compute objective gradient objective_gradient = self.objective.gradient(model, batch, batch_labels) # Multiply each param by its given step_size model.mul_step_size(step_size, objective_gradient) # Update the model by subtracting the objective-gradient (descent) from the # current models variables, e.g., (prototypes, omega) in case of GMLVQ model.set_variables( np.subtract( # returns out=objective_gradient model.get_variables(), objective_gradient, out=objective_gradient, ) ) # Call the callback function if provided with updated values. if self.callback is not None: state = _update_state( STATE_KEYS, variables=np.copy(model.get_variables()), nit=i_run + 1, fun=self.objective(model, data, labels), step_size=step_size, ) # Simply return (stop the solver process) when callback returns true. if self.callback(state): return .. GENERATED FROM PYTHON SOURCE LINES 125-130 The CustomSteepestGradientDescent above, accompanied with some tests and documentation, would make a great addition to the sklvq package. However, it can also directly be passed to the algorithm. Some other solvers might require more functionality not supported by the models, this can be added dynamically to the model instances or by extending the required model and creating a custom model class. .. GENERATED FROM PYTHON SOURCE LINES 130-147 .. code-block:: Python data, labels = load_iris(return_X_y=True) model = GLVQ( solver_type=CustomSteepestGradientDescent, distance_type="squared-euclidean", activation_type="sigmoid", activation_params={"beta": 2}, ) model.fit(data, labels) # Predict the labels using the trained model predicted_labels = model.predict(data) # Print a classification report (sklearn) print(classification_report(labels, predicted_labels)) .. rst-class:: sphx-glr-script-out .. code-block:: none precision recall f1-score support 0 1.00 1.00 1.00 50 1 0.92 0.94 0.93 50 2 0.94 0.92 0.93 50 accuracy 0.95 150 macro avg 0.95 0.95 0.95 150 weighted avg 0.95 0.95 0.95 150 .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 0.144 seconds) .. _sphx_glr_download_auto_examples_05_customization_plot_04_solvers.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_04_solvers.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_04_solvers.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: plot_04_solvers.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_