sklvq.models.GMLVQ

class sklvq.models.GMLVQ(distance_type: Union[str, type] = 'adaptive-squared-euclidean', distance_params: Optional[dict] = None, activation_type: Union[str, type] = 'sigmoid', activation_params: Optional[dict] = None, discriminant_type: Union[str, type] = 'relative-distance', discriminant_params: Optional[dict] = None, solver_type: Union[str, type] = 'steepest-gradient-descent', solver_params: Optional[dict] = None, prototype_init: Union[str, numpy.ndarray] = 'class-conditional-mean', prototype_n_per_class: Union[int, numpy.ndarray] = 1, relevance_init='identity', relevance_normalization: bool = True, relevance_n_components: Union[str, int] = 'all', relevance_regularization: Union[int, float] = 0, random_state: Optional[Union[int, numpy.random.mtrand.RandomState]] = None, force_all_finite: Union[str, bool] = True)

Generalized Matrix Learning Vector Quantization

This model uses the sklvq.objectives.GeneralizedLearningObjective as its objective function [1]. In addition to learning the positions of the prototypes it learns a relevance matrix that is used in the distance functions [2].

Parameters

distance_type{“adaptive-squared-euclidean”} or Class, default=”squared-euclidean”

Distance function that employs a relevance matrix in its calculation.

• “adaptive-squared-euclidean”

  See sklvq.distances.AdaptiveSquaredEuclidean

distance_paramsDict, default=None

Parameters passed to init of distance callable

activation_type{“identity”, “sigmoid”, “soft+”, “swish”} or Class, default=”sigmoid”

Parameters passed to init of activation function. See the documentation of the activation functions for parameters and defaults.

• “identity”

  See sklvq.activations.Identity

• “sigmoid”

  See sklvq.activations.Sigmoid

• “soft+”

  See sklvq.activations.SoftPlus

• “swish”

  See sklvq.activations.Swish

activation_paramsDict, default=None

Parameters passed to init of activation function. See the documentation of activation functions for function dependent parameters and defaults.

discriminant_type{“relative-distance”} or Class, default = “relative-distance”

The discriminant function. Note that different discriminant type may require to rewrite the decision_function and predict_proba methods.

• “relative-distance”

  See sklvq.discriminants.RelativeDistance

discriminant_paramsDict, default=None

Parameters passed to init of discriminant callable. See the documentation of the discriminant functions for parameters and defaults.

solver_type{“sgd”, “wgd”, “adam”, “lbfgs”, “bfgs”},

The solver used for optimization

• “sgd” or “steepest-gradient-descent”

  See sklvq.solvers.SteepestGradientDescent.

• “wgd” or “waypoint-gradient-descent”

  See sklvq.solvers.WaypointGradientDescent.

• “adam” or “adaptive-moment-estimation”

  See sklvq.solvers.AdaptiveMomentEstimation.

• “bfgs” or “broyden-fletcher-goldfarb-shanno”

  Implementation from scipy package.

• “lbfgs” or “limited-memory-bfgs”

  Implementation from scipy package.

solver_paramsdict, default=None

Parameters passed to init of solvers. See the documentation of the solvers relevant parameters and defaults.

prototype_init: “class-conditional-mean” or ndarray, default=”class-conditional-mean”

Default will initiate the prototypes to the class conditional mean with a small random offset. Custom numpy array can be passed to change the initial positions of the prototypes.

prototype_n_per_class: int or np.ndarray, optional, default=1

Default will generate single prototype per class. In the case of unequal number of prototypes per class is needed, provide this as np.ndarray. For example, prototype_n_per_class = np.array([1, 6, 3]) this will result in one prototype for the first class, six for the second, and three for the third. Note that the order needs to be the same as the on in the classes_ attribute, which is equal to calling np.unique(labels).

relevance_init{“identity”, “random”} or np.ndarray, default=”identity”

Default will initiate the omega matrices to be the identity matrix. The rank of the matrix can be reduced by setting the relevance_n_components attribute [3].

relevance_normalization: bool, optional, default=True

Flag to indicate whether to normalize omega, whenever it is updated, such that the trace of the relevance matrix is equal to 1.

relevance_n_components: str {“all”} or int, optional, default=”all”

For a square relevance matrix use the string “all” (default). For a rectangular relevance matrix use set the number of components explicitly by providing it as an int.

random_stateint, RandomState instance, default=None

Set the random number generation for reproducibility purposes. Used in random offset of prototypes and shuffling of the data in the solvers. Potentially, also used in the random generation of relevance matrix.

force_all_finite{True, “allow-nan”}, default=True

Whether to raise an error on np.inf, np.nan, pd.NA in array. The possibilities are:

• True: Force all values of array to be finite.

• “allow-nan”: accepts only np.nan and pd.NA values in array. Values cannot be infinite.

References

[1] Sato, A., and Yamada, K. (1996) “Generalized Learning Vector Quantization.” Advances in Neural Network Information Processing Systems, 423–429, 1996.

[2] Schneider, P., Biehl, M., & Hammer, B. (2009). “Adaptive Relevance Matrices in Learning Vector Quantization” Neural Computation, 21(12), 3532–3561, 2009.

[3] Bunte, K., Schneider, P., Hammer, B., Schleif, F.-M., Villmann, T., & Biehl, M. (2012). “Limited Rank Matrix Learning, discriminative dimension reduction and visualization.” Neural Networks, 26, 159–173, 2012.

Attributes

classes_ndarray of shape (n_classes,)

Class labels for each output.

prototypes_ndarray of shape (n_protoypes, n_features)

Positions of the prototypes after fit(X, labels) has been called.

prototypes_labels_ndarray of shape (n_prototypes)

Labels for each prototypes. Labels are indexes to classes_

omega_: ndarray with size depending on initialization, default (n_features, n_features)

Omega matrix that was found during training and defines the relevance matrix lambda_.

lambda_: ndarray of size (n_features, n_features)

The relevance matrix omega_.T.dot(omega_)

omega_hat_: ndarray

The omega matrix found by the eigenvalue decomposition of the relevance matrix lambda_. The eigenvectors (columns of omega_hat_) can be used to transform the X [3].

eigenvalues_: ndarray

The corresponding eigenvalues to omega_hat_ found by the eigenvalue decomposition of the relevance matrix lambda_

__init__(distance_type: Union[str, type] = 'adaptive-squared-euclidean', distance_params: Optional[dict] = None, activation_type: Union[str, type] = 'sigmoid', activation_params: Optional[dict] = None, discriminant_type: Union[str, type] = 'relative-distance', discriminant_params: Optional[dict] = None, solver_type: Union[str, type] = 'steepest-gradient-descent', solver_params: Optional[dict] = None, prototype_init: Union[str, numpy.ndarray] = 'class-conditional-mean', prototype_n_per_class: Union[int, numpy.ndarray] = 1, relevance_init='identity', relevance_normalization: bool = True, relevance_n_components: Union[str, int] = 'all', relevance_regularization: Union[int, float] = 0, random_state: Optional[Union[int, numpy.random.mtrand.RandomState]] = None, force_all_finite: Union[str, bool] = True)

Initialize self. See help(type(self)) for accurate signature.

add_partial_gradient(gradient, partial_gradient, i_prototype) → None

Adds the partial gradient to the correct part of the gradient, which depends on i_prototype.

Parameters

gradientndarray

Same shape as the get_variables() would return.

partial_gradientndarray

1d array containing the partial gradient.

i_prototypeint

The index of the prototype to which the partial gradient was computed.

decision_function(X: numpy.ndarray)

Evaluates the decision function for the samples in X. Shape for binary class is (n_observations,) with the decision values for the “greater” class. In the multiclass case it returns decision values for each class and therefore has the shape (n_observations, n_classes).

Parameters

Xndarray

The data.

Returns

decision_valuesndarray

Binary case shape is (n_observations,) and the multiclass case (n_observations, n_classes)

fit(X: numpy.ndarray, y: numpy.ndarray)

Fit function that provides the general implementation of the LVQ algorithms. It checks the data, calls

before_fit method, calls the solve method of the solver, and the after_fit method.

Parameters

Xndarray of shape (number of observations, number of dimensions)

yndarray of size (number of observations)

Returns

self

The trained model

fit_transform(data: numpy.ndarray, y: numpy.ndarray, **transform_params) → numpy.ndarray

Parameters

datandarray with shape (n_samples, n_features)

Data used for fit and that will be transformed.

ynp.ndarray with length (n_samples)

Labels corresponding to the X samples.

transform_params :

Parameters passed to transform function

Returns

The data projected on columns of omega_hat_ with shape (n_samples, n_columns)

get_model_params() → Tuple[numpy.ndarray, numpy.ndarray]

Returns a tuple of all model parameters. In this case the prototypes and omega matrix.

Returns

ndarray

Returns a tuple of views, i.e., the prototypes and omega matrix.

get_omega()

Convenience function to return self.omega_

Returns

ndarray, with shape depending on initialization of omega.

get_params(deep=True)

Get parameters for this estimator.

Parameters

deepbool, default=True

If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns

paramsdict

Parameter names mapped to their values.

get_prototypes() → numpy.ndarray

Return a view into self._variables of the the shape of the prototypes (n_prototypes, n_features). At the moment only consistency function, does not actually create the shape and only works after self.prototypes_ has been set.

Returns

prototypesndarray of shape (n_prototypes, n_features)

View into self._variables with shape specified above.

get_variables() → numpy.ndarray

Returns the self._variables array that owns the memory allocated for the model parameters.

Returns

_variablesndarray

returns the model’s _variables array.

mul_step_size(step_sizes: Union[int, float, numpy.ndarray], gradient: numpy.ndarray) → None

If step sizes is a scalar value just multiplies the gradient with the step size. If it is an array (with same length as number of model parameters) each model parameter is multiplied by its own step size.

Parameters

step_sizesfloat or ndarray

The scalar or list of values containing the step sizes.

gradientndarray

Same shape as the get_variables() would return.

normalize_variables(var_buffer: numpy.ndarray) → None

Modifies the var_buffer as if it was the variables array provided by get_variables(). Will select, reshape and normalize the correct parts of the variable buffer.

Parameters

var_bufferndarray

Array with the same size as the model’s variables array as returned by get_variables().

predict(X: numpy.ndarray)

Predict function

The decision is made for the label of the prototype with the minimum decision value, as provided by the decision_function().

Parameters

Xndarray

The data.

Returns

ndarray of shape (n_observations)

Returns the predicted labels.

predict_proba(X: numpy.ndarray)

Parameters

Xndarray

The data.

Returns

confidence_scoresndarray of shape (n_observations, n_classes)

score(X, y, sample_weight=None)

Return the mean accuracy on the given test data and labels.

In multi-label classification, this is the subset accuracy which is a harsh metric since you require for each sample that each label set be correctly predicted.

Parameters

Xarray-like of shape (n_samples, n_features)

Test samples.

yarray-like of shape (n_samples,) or (n_samples, n_outputs)

True labels for X.

sample_weightarray-like of shape (n_samples,), default=None

Sample weights.

Returns

scorefloat

Mean accuracy of self.predict(X) wrt. y.

set_model_params(new_model_params: Tuple[numpy.ndarray, numpy.ndarray])

Changes the model’s internal parameters. Copies the values of model_params into self.prototypes_ and self.omega_ therefor updating the self.variables_ array.

Also normalized the relevance matrix if necessary.

Parameters

new_model_paramstuple of ndarrays

Shapes depend on initialization but in the case of a square relevance matrix: tuple((n_prototypes, n_features), (n_features, n_features))

set_omega(omega)

Convenience function that makes sure to copy the value to self.omega_ and not overwrite it.

Parameters

omegandarray with same shape as self.omega_

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters

**paramsdict

Estimator parameters.

Returns

selfestimator instance

Estimator instance.

set_prototypes(new_prototypes: numpy.ndarray) → None

Accepts a new_prototypes array with the same shape as self.prototypes_ and overwrites the self._variables array by copying the values of the new_prototypes.

Parameters

new_prototypesndarray of shape (n_prototypes, n_features)

The new prototypes the model should store.

set_variables(new_variables: numpy.ndarray) → None

Modifies the self._variables by copying the values of new_variables into the memory of self._variables.

Parameters

new_variablesndarray

1d numpy array that contains all the model parameters in continuous memory

to_model_params_view(var_buffer: numpy.ndarray) → Tuple[numpy.ndarray, numpy.ndarray]

Parameters

var_bufferndarray

Array with the same size as the model’s variables array as returned by get_variables().

Returns

tuple

Returns a tuple with the prototypes and omega matrix as ndarrays.

to_omega(var_buffer: numpy.ndarray) → numpy.ndarray

Returns a view (of the shape of the model’s omega) into the provided variables buffer of the same size as the model’s variables array.

Parameters

var_bufferndarray

Array with the same size as the model’s variables array as returned by get_variables().

Returns

ndarray

Shape depending on initialization but in case of a square matrix (n_features, n_features.

to_prototypes_view(var_buffer: numpy.ndarray) → numpy.ndarray

Returns a view (of the shape of the model’s prototypes) into the provided variables buffer of the same size as the model’s variables array.

Parameters

var_bufferndarray

Array with the same size as the model’s variables array as returned by get_variables().

Returns

ndarray of shape (n_prototypes, n_features)

Prototype view into the var_buffer.

transform(X: numpy.ndarray, scale: bool = False) → numpy.ndarray

Parameters

Xnp.ndarray with shape (n_samples, n_features)

Data that needs to be transformed

scale{True, False}, default = False

Controls if the eigenvectors the data is projected on are scaled by the square root of their eigenvalues.

Returns

The data projected on columns of omega_hat_ with shape (n_samples, n_columns)