sklvq.models.GMLVQ

class sklvq.models.GMLVQ(distance_type: str | type = 'adaptive-squared-euclidean', distance_params: dict | None = None, activation_type: str | type = 'sigmoid', activation_params: dict | None = None, discriminant_type: str | type = 'relative-distance', discriminant_params: dict | None = None, solver_type: str | type = 'steepest-gradient-descent', solver_params: dict | None = None, prototype_init: str | ndarray = 'class-conditional-mean', prototype_n_per_class: int | ndarray = 1, relevance_init='identity', relevance_normalization: bool = True, relevance_correction: ndarray = None, relevance_n_components: str | int = 'all', relevance_regularization: float = 0, random_state: int | RandomState = None, force_all_finite: str | bool = True)[source]

Generalized Matrix Learning Vector Quantization

This model uses the sklvq.objectives.GeneralizedLearningObjective as its objective function [1]. In addition to learning the positions of the prototypes it learns a relevance matrix that is used in the distance functions [2].

Parameters:
distance_type{“adaptive-squared-euclidean”} or Class, default=”squared-euclidean”

Distance function that employs a relevance matrix in its calculation.

distance_paramsDict, default=None

Parameters passed to init of distance callable

activation_type{“identity”, “sigmoid”, “soft+”, “swish”} or Class, default=”sigmoid”

Parameters passed to init of activation function. See the documentation of the activation functions for parameters and defaults.

activation_paramsDict, default=None

Parameters passed to init of activation function. See the documentation of activation functions for function dependent parameters and defaults.

discriminant_type{“relative-distance”} or Class, default = “relative-distance”

The discriminant function. Note that different discriminant type may require to rewrite the decision_function and predict_proba methods.

discriminant_paramsDict, default=None

Parameters passed to init of discriminant callable. See the documentation of the discriminant functions for parameters and defaults.

solver_type{“sgd”, “wgd”, “adam”, “lbfgs”, “bfgs”},

The solver used for optimization

solver_paramsdict, default=None

Parameters passed to init of solvers. See the documentation of the solvers relevant parameters and defaults.

prototype_init: “class-conditional-mean” or ndarray, default=”class-conditional-mean”

Default will initiate the prototypes to the class conditional mean with a small random offset. Custom numpy array can be passed to change the initial positions of the prototypes.

prototype_n_per_class: int or np.ndarray, optional, default=1

Default will generate single prototype per class. In the case of unequal number of prototypes per class is needed, provide this as np.ndarray. For example, prototype_n_per_class = np.array([1, 6, 3]) this will result in one prototype for the first class, six for the second, and three for the third. Note that the order needs to be the same as the on in the classes_ attribute, which is equal to calling np.unique(labels).

relevance_init{“identity”, “random”} or np.ndarray, default=”identity”

Default will initiate the omega matrices to be the identity matrix. The rank of the matrix can be reduced by setting the relevance_n_components attribute [3].

relevance_normalization: bool, optional, default=True

Flag to indicate whether to normalize omega, whenever it is updated, such that the trace of the relevance matrix is equal to 1.

relevance_correction: np.ndarray, optional

Matrix that will be used to project out any contribution from unwanted directions in omega_ during training. When set this will be applied every update step before the relevance_normalization is applied. The correction matrix is responsible for a mapping where the unwanted directions are removed [4].

relevance_n_components: str {“all”} or int, optional, default=”all”

For a square relevance matrix use the string “all” (default). For a rectangular relevance matrix use set the number of components explicitly by providing it as an int.

random_stateint, RandomState instance, default=None

Set the random number generation for reproducibility purposes. Used in random offset of prototypes and shuffling of the data in the solvers. Potentially, also used in the random generation of relevance matrix.

force_all_finite{True, “allow-nan”}, default=True

Whether to raise an error on np.inf, np.nan, pd.NA in array. The possibilities are:

  • True: Force all values of array to be finite.

  • “allow-nan”: accepts only np.nan and pd.NA values in array. Values cannot be infinite.

Attributes:
classes_ndarray of shape (n_classes,)

Class labels for each output.

prototypes_ndarray of shape (n_protoypes, n_features)

Positions of the prototypes after fit(X, labels) has been called.

prototypes_labels_ndarray of shape (n_prototypes)

Labels for each prototypes. Labels are indexes to classes_

omega_: ndarray with size depending on initialization, default (n_features, n_features)

Omega matrix that was found during training and defines the relevance matrix lambda_.

lambda_: ndarray of size (n_features, n_features)

The relevance matrix omega_.T.dot(omega_)

omega_hat_: ndarray

The eigenvectors_ scaled by the square root of the eigenvalues_ [3].

eigenvectors_: ndarray

The eigenvectors found by the eigenvalue decomposition of the relevance matrix lambda_

eigenvalues_: ndarray

The corresponding eigenvalues to eigenvectors_ found by the eigenvalue decomposition of the relevance matrix lambda_

References

[1] Sato, A., and Yamada, K. (1996) “Generalized Learning Vector Quantization.” Advances in Neural Network Information Processing Systems, 423-429, 1996.

[2] Schneider, P., Biehl, M., & Hammer, B. (2009). “Adaptive Relevance Matrices in Learning Vector Quantization” Neural Computation, 21(12), 3532-3561, 2009.

[3] Bunte, K., Schneider, P., Hammer, B., Schleif, F.-M., Villmann, T., & Biehl, M. (2012). “Limited Rank Matrix Learning, discriminative dimension reduction and visualization.” Neural Networks, 26, 159–173, 2012.

[4] van Veen, R., Tamboli, N. R. B., Lövdal, S., Meles, S. K., Renken, R. J., de Vries, G.-J., Arnaldi, D., Morbelli, S., Clavero, P., Obeso, J. A., Oroz, M. C. R., Leenders, K. L., Villmann, T., & Biehl, M. (2024). Subspace corrected relevance learning with application in neuroimaging. Artificial Intelligence in Medicine, 149, 102786, 2024. Elsevier BV.

__init__(distance_type: str | type = 'adaptive-squared-euclidean', distance_params: dict | None = None, activation_type: str | type = 'sigmoid', activation_params: dict | None = None, discriminant_type: str | type = 'relative-distance', discriminant_params: dict | None = None, solver_type: str | type = 'steepest-gradient-descent', solver_params: dict | None = None, prototype_init: str | ndarray = 'class-conditional-mean', prototype_n_per_class: int | ndarray = 1, relevance_init='identity', relevance_normalization: bool = True, relevance_correction: ndarray = None, relevance_n_components: str | int = 'all', relevance_regularization: float = 0, random_state: int | RandomState = None, force_all_finite: str | bool = True)[source]
add_partial_gradient(gradient, partial_gradient, i_prototype) → None[source]

Adds the partial gradient to the correct part of the gradient, which depends on i_prototype.

Parameters:
gradientndarray

Same shape as the get_variables() would return.

partial_gradientndarray

1d array containing the partial gradient.

i_prototypeint

The index of the prototype to which the partial gradient was computed.

decision_function(X: ndarray)

Evaluates the decision function for the samples in X. Shape for binary class is (n_observations,) with the decision values for the “greater” class. In the multiclass case it returns decision values for each class and therefore has the shape (n_observations, n_classes).

Parameters:
Xndarray

The data.

Returns:
decision_valuesndarray

Binary case shape is (n_observations,) and the multiclass case (n_observations, n_classes)

fit(X: ndarray, y: ndarray)
Fit function that provides the general implementation of the LVQ algorithms. It checks the data, calls

before_fit method, calls the solve method of the solver, and the after_fit method.

Parameters:
Xndarray of shape (number of observations, number of dimensions)
yndarray of size (number of observations)
Returns:
self

The trained model

fit_transform(data: ndarray, y: ndarray, **transform_params) → ndarray[source]
Parameters:
datandarray with shape (n_samples, n_features)

Data used for fit and that will be transformed.

ynp.ndarray with length (n_samples)

Labels corresponding to the X samples.

transform_params

Parameters passed to transform function

Returns:
The data projected on columns of omega_hat_ with shape (n_samples, n_columns)
get_metadata_routing()

Get metadata routing of this object.

Please check User Guide on how the routing mechanism works.

Returns:
routingMetadataRequest

A MetadataRequest encapsulating routing information.

get_model_params() → Tuple[ndarray, ndarray][source]

Returns a tuple of all model parameters. In this case the prototypes and omega matrix.

Returns:
ndarray

Returns a tuple of views, i.e., the prototypes and omega matrix.

get_omega()[source]

Convenience function to return self.omega_

Returns:
ndarray, with shape depending on initialization of omega.
get_params(deep=True)

Get parameters for this estimator.

Parameters:
deepbool, default=True

If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:
paramsdict

Parameter names mapped to their values.

get_prototypes() → ndarray

Return a view into self._variables of the the shape of the prototypes (n_prototypes, n_features). At the moment only consistency function, does not actually create the shape and only works after self.prototypes_ has been set.

Returns:
prototypesndarray of shape (n_prototypes, n_features)

View into self._variables with shape specified above.

get_variables() → ndarray

Returns the self._variables array that owns the memory allocated for the model parameters.

Returns:
_variablesndarray

returns the model’s _variables array.

mul_step_size(step_sizes: float | ndarray, gradient: ndarray) → None[source]

If step sizes is a scalar value just multiplies the gradient with the step size. If it is an array (with same length as number of model parameters) each model parameter is multiplied by its own step size.

Parameters:
step_sizesfloat or ndarray

The scalar or list of values containing the step sizes.

gradientndarray

Same shape as the get_variables() would return.

normalize_variables(var_buffer: ndarray) → None[source]

Modifies the var_buffer as if it was the variables array provided by get_variables(). Will select, reshape and normalize the correct parts of the variable buffer.

Parameters:
var_bufferndarray

Array with the same size as the model’s variables array as returned by get_variables().

predict(X: ndarray, threshold: float = 0.5)

Predict function

The decision is made for the label of the prototype with the minimum decision value, as provided by the decision_function(). To choose a different cutoff point (e.g., based upon ROC analysis for 2-class data), threshold can be set, which acts on the values from predict_proba(). The threshold value won’t be used when the model has been fitted to data with more than 2 classes.

Parameters:
Xndarray

The data.

thresholdfloat

threshold for predict_proba() values (default 0.5), should be in the interval [0,1]

Returns:
ndarray of shape (n_observations)

Returns the predicted labels.

predict_proba(X: ndarray)
Parameters:
Xndarray

The data.

Returns:
confidence_scoresndarray of shape (n_observations, n_classes)
score(X, y, sample_weight=None)

Return the mean accuracy on the given test data and labels.

In multi-label classification, this is the subset accuracy which is a harsh metric since you require for each sample that each label set be correctly predicted.

Parameters:
Xarray-like of shape (n_samples, n_features)

Test samples.

yarray-like of shape (n_samples,) or (n_samples, n_outputs)

True labels for X.

sample_weightarray-like of shape (n_samples,), default=None

Sample weights.

Returns:
scorefloat

Mean accuracy of self.predict(X) w.r.t. y.

set_model_params(new_model_params: Tuple[ndarray, ndarray])[source]

Changes the model’s internal parameters. Copies the values of model_params into self.prototypes_ and self.omega_ therefor updating the self.variables_ array.

Also normalized the relevance matrix if necessary.

Parameters:
new_model_paramstuple of ndarrays

Shapes depend on initialization but in the case of a square relevance matrix: tuple((n_prototypes, n_features), (n_features, n_features))

set_omega(omega)[source]

Convenience function that makes sure to copy the value to self.omega_ and not overwrite it.

Parameters:
omegandarray with same shape as self.omega_
set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:
**paramsdict

Estimator parameters.

Returns:
selfestimator instance

Estimator instance.

set_predict_request(*, threshold: bool | None | str = '$UNCHANGED$') → GMLVQ

Request metadata passed to the predict method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to predict if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to predict.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:
thresholdstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for threshold parameter in predict.

Returns:
selfobject

The updated object.

set_prototypes(new_prototypes: ndarray) → None

Accepts a new_prototypes array with the same shape as self.prototypes_ and overwrites the self._variables array by copying the values of the new_prototypes.

Parameters:
new_prototypesndarray of shape (n_prototypes, n_features)

The new prototypes the model should store.

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → GMLVQ

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to score.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:
sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in score.

Returns:
selfobject

The updated object.

set_transform_request(*, scale: bool | None | str = '$UNCHANGED$') → GMLVQ

Request metadata passed to the transform method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to transform if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to transform.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:
scalestr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for scale parameter in transform.

Returns:
selfobject

The updated object.

set_variables(new_variables: ndarray) → None[source]

Modifies the self._variables by copying the values of new_variables into the memory of self._variables.

Parameters:
new_variablesndarray

1d numpy array that contains all the model parameters in continuous memory

to_model_params_view(var_buffer: ndarray) → Tuple[ndarray, ndarray][source]
Parameters:
var_bufferndarray

Array with the same size as the model’s variables array as returned by get_variables().

Returns:
tuple

Returns a tuple with the prototypes and omega matrix as ndarrays.

to_omega(var_buffer: ndarray) → ndarray[source]

Returns a view (of the shape of the model’s omega) into the provided variables buffer of the same size as the model’s variables array.

Parameters:
var_bufferndarray

Array with the same size as the model’s variables array as returned by get_variables().

Returns:
ndarray

Shape depending on initialization but in case of a square matrix (n_features, n_features.

to_prototypes_view(var_buffer: ndarray) → ndarray[source]

Returns a view (of the shape of the model’s prototypes) into the provided variables buffer of the same size as the model’s variables array.

Parameters:
var_bufferndarray

Array with the same size as the model’s variables array as returned by get_variables().

Returns:
ndarray of shape (n_prototypes, n_features)

Prototype view into the var_buffer.

transform(X: ndarray, scale: bool = False) → ndarray[source]
Parameters:
Xnp.ndarray with shape (n_samples, n_features)

Data that needs to be transformed

scale{True, False}, default = False

Controls if the eigenvectors the data is projected on are scaled by the square root of their eigenvalues.

Returns:
The data projected on columns of omega_hat_ with shape (n_samples, n_columns)