sklvq.models.LGMLVQ

class sklvq.models.LGMLVQ(distance_type: Union[str, type] = 'local-adaptive-squared-euclidean', distance_params: Optional[dict] = None, activation_type: Union[str, type] = 'identity', activation_params: Optional[dict] = None, discriminant_type: Union[str, type] = 'relative-distance', discriminant_params: Optional[dict] = None, solver_type: Union[str, type] = 'steepest-gradient-descent', solver_params: Optional[dict] = None, prototype_init: Union[str, numpy.ndarray] = 'class-conditional-mean', prototype_n_per_class: [<class 'int'>, <class 'numpy.ndarray'>] = 1, relevance_init: Union[str, numpy.ndarray] = 'identity', relevance_normalization: bool = True, relevance_n_components: Union[str, int] = 'all', relevance_localization: str = 'prototypes', random_state: Optional[Union[int, numpy.random.mtrand.RandomState]] = None, force_all_finite: Union[str, int] = True)

Localized Generalized Matrix Learning Vector Quantization

This model uses the sklvq.objectives.GeneralizedLearningObjective as its objective function [1]. In addition to learning the positions of the prototypes it learns a set of relevance matrices in a localized manner, that are used in the distance functions [2].

Parameters

distance_type“local-adaptive-squared-euclidean” or Class

Distance function that employs multiple relevance matrix in its calculation. This is controlled by the localization setting.

• “local-adaptive-squared-euclidean”

  See sklvq.distances.LocalAdaptiveSquaredEuclidean

distance_paramsdict, default=None

Parameters passed to init of distance class.

activation_type{“identity”, “sigmoid”, “soft+”, “swish”} or Class, default=”sigmoid”

The activation function used in the objective function. Can be any of the activation function in the list or custom class.

• “identity”

  See sklvq.activations.Identity

• “sigmoid”

  See sklvq.activations.Sigmoid

• “soft+”

  See sklvq.activations.SoftPlus

• “swish”

  See sklvq.activations.Swish

activation_paramsDict, default=None

Parameters passed to init of activation function. See the documentation of the activation functions for parameters and defaults.

discriminant_type“relative-distance” or Class

The discriminant function. Note that different discriminant type may require to rewrite the decision_function and predict_proba methods.

• “relative-distance”

  See sklvq.discriminants.RelativeDistance

discriminant_paramsDict, default=None

Parameters passed to init of discriminant callable. See the documentation of the discriminant functions for parameters and defaults.

solver_type{“sgd”, “wgd”, “adam”, “lbfgs”, “bfgs”},

The solver used for optimization

• “sgd” or “steepest-gradient-descent”

  See sklvq.solvers.SteepestGradientDescent.

• “wgd” or “waypoint-gradient-descent”

  See sklvq.solvers.WaypointGradientDescent.

• “adam” or “adaptive-moment-estimation”

  See sklvq.solvers.AdaptiveMomentEstimation.

• “bfgs” or “broyden-fletcher-goldfarb-shanno”

  Implementation from scipy package.

• “lbfgs” or “limited-memory-bfgs”

  Implementation from scipy package.

solver_paramsdict, default=None

Parameters passed to init of solvers. See the documentation of the solvers relevant parameters and defaults.

prototype_init: “class-conditional-mean” or ndarray, default=”class-conditional-mean”

Default will initiate the prototypes to the class conditional mean with a small random offset. Custom numpy array can be passed to change the initial positions of the prototypes.

prototype_n_per_class: int or np.ndarray, optional, default=1

Default will generate single prototype per class. In the case of unequal number of prototypes per class is needed, provide this as np.ndarray. For example, prototype_n_per_class = np.array([1, 6, 3]) this will result in one prototype for the first class, six for the second, and three for the third. Note that the order needs to be the same as the on in the classes_ attribute, which is equal to calling np.unique(labels).

relevance_init{“identity”, “random”} or np.ndarray, default=”identity”

Default will initiate the omega matrices to be the identity matrix. The rank of the matrix can be reduced by

setting the relevance_n_components attribute [3].

relevance_normalization: bool, optional, default=True

Flag to indicate whether to normalize omega, whenever it is updated, such that the trace of the relevance matrix is equal to 1.

relevance_n_components: str {“all”} or int, optional, default=”all”

For a square relevance matrix use the string “all” (default). For a rectangular relevance matrix use set the number of components explicitly by providing it as an int.

relevance_localization: {“prototypes”, “class”}, default=”prototypes”

Setting that controls the localization of the relevance matrices. Either per prototype, where each prototype has its own relevance matrix. Or per class where each class has its own relevance matrix. Note that when one prototype per class is used, changing this setting has no effect.

random_stateint, RandomState instance, default=None

Set the random number generation for reproducibility purposes. Used in random offset of prototypes and shuffling of the data in the solvers. Potentially, also used in the random generation of relevance matrix.

force_all_finite{True, “allow-nan”}, default=True

Whether to raise an error on np.inf, np.nan, pd.NA in array. The possibilities are:

• True: Force all values of array to be finite.

• “allow-nan”: accepts only np.nan and pd.NA values in array. Values cannot be infinite.

References

[1] Sato, A., and Yamada, K. (1996) “Generalized Learning Vector Quantization.” Advances in Neural Network Information Processing Systems, 423–429, 1996.

[2] Schneider, P., Biehl, M., & Hammer, B. (2009). “Adaptive Relevance Matrices in Learning Vector Quantization” Neural Computation, 21(12), 3532–3561, 2009.

[3] Bunte, K., Schneider, P., Hammer, B., Schleif, F.-M., Villmann, T., & Biehl, M. (2012). “Limited Rank Matrix Learning, discriminative dimension reduction and visualization.” Neural Networks, 26, 159–173, 2012.

Attributes

classes_ndarray of shape (n_classes,)

Class labels for each output.

prototypes_ndarray of shape (n_protoypes, n_features)

Positions of the prototypes after fit(X, labels) has been called.

prototypes_labels_ndarray of shape (n_prototypes)

Labels for each prototypes. Labels are indexes to classes_

omega_: ndarray with size (n_matrices, n_features, n_features)

omega_ matrices that were found during training and define the relevance matrices lambda_.

lambda_: ndarray of size (n_matrices, n_features, n_features)

The relevance matrices omega_.T.dot(omega_) per matrix.

omega_hat_: ndarray

The omega matrices found by the eigenvalue decomposition of the relevance matrices lambda_. The eigenvectors (columns of omega_hat_) can be used to transform the data

[3]. This results in multiple possible transformations, one per relevance matrix.

eigenvalues_: ndarray

The corresponding eigenvalues to omega_hat_ found by the eigenvalue decomposition of the relevance matrices lambda_

__init__(distance_type: Union[str, type] = 'local-adaptive-squared-euclidean', distance_params: Optional[dict] = None, activation_type: Union[str, type] = 'identity', activation_params: Optional[dict] = None, discriminant_type: Union[str, type] = 'relative-distance', discriminant_params: Optional[dict] = None, solver_type: Union[str, type] = 'steepest-gradient-descent', solver_params: Optional[dict] = None, prototype_init: Union[str, numpy.ndarray] = 'class-conditional-mean', prototype_n_per_class: [<class 'int'>, <class 'numpy.ndarray'>] = 1, relevance_init: Union[str, numpy.ndarray] = 'identity', relevance_normalization: bool = True, relevance_n_components: Union[str, int] = 'all', relevance_localization: str = 'prototypes', random_state: Optional[Union[int, numpy.random.mtrand.RandomState]] = None, force_all_finite: Union[str, int] = True)

Initialize self. See help(type(self)) for accurate signature.

add_partial_gradient(gradient, partial_gradient, i_prototype) → None

Adds the partial gradient to the correct part of the gradient, which depends on i_prototype.

Parameters

gradientndarray

Same shape as the get_variables() would return.

partial_gradientndarray

1d array containing the partial gradient.

i_prototypeint

The index of the prototype to which the partial gradient was computed.

decision_function(X: numpy.ndarray)

Evaluates the decision function for the samples in X. Shape for binary class is (n_observations,) with the decision values for the “greater” class. In the multiclass case it returns decision values for each class and therefore has the shape (n_observations, n_classes).

Parameters

Xndarray

The data.

Returns

decision_valuesndarray

Binary case shape is (n_observations,) and the multiclass case (n_observations, n_classes)

fit(X: numpy.ndarray, y: numpy.ndarray)

Fit function that provides the general implementation of the LVQ algorithms. It checks the data, calls

before_fit method, calls the solve method of the solver, and the after_fit method.

Parameters

Xndarray of shape (number of observations, number of dimensions)

yndarray of size (number of observations)

Returns

self

The trained model

fit_transform(X: numpy.ndarray, y: numpy.ndarray, **trans_params) → numpy.ndarray

Parameters

Xndarray with shape (n_samples, n_features)

Data used for fit and that will be transformed.

ynp.ndarray with length (n_samples)

Labels corresponding to the X samples.

trans_params :

Parameters passed to transform function

Returns

The data projected on columns of omega_hat_ with shape (n_matrices, n_samples,

n_columns)

get_model_params() → Tuple[numpy.ndarray, numpy.ndarray]

Returns a tuple of all model parameters. In this case the prototypes and omega matrix.

Returns

ndarray

Returns a tuple of views, i.e., the prototypes and omega matrix.

get_omega()

Function to return self.omega_ (consistency)

Returns

ndarray, with shape depending on initialization of omega.

get_params(deep=True)

Get parameters for this estimator.

Parameters

deepbool, default=True

If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns

paramsdict

Parameter names mapped to their values.

get_prototypes() → numpy.ndarray

Return a view into self._variables of the the shape of the prototypes (n_prototypes, n_features). At the moment only consistency function, does not actually create the shape and only works after self.prototypes_ has been set.

Returns

prototypesndarray of shape (n_prototypes, n_features)

View into self._variables with shape specified above.

get_variables() → numpy.ndarray

Returns the self._variables array that owns the memory allocated for the model parameters.

Returns

_variablesndarray

returns the model’s _variables array.

mul_step_size(step_sizes: Union[int, float, numpy.ndarray], gradient: numpy.ndarray) → None

If step sizes is a scalar value just multiplies the gradient with the step size. If it is an array (with same length as number of model parameters) each model parameter is multiplied by its own step size.

Parameters

step_sizesfloat or ndarray

The scalar or list of values containing the step sizes.

gradientndarray

Same shape as the get_variables() would return.

normalize_variables(var_buffer: numpy.ndarray) → None

Modifies the var_buffer as if it was the variables array provided by get_variables(). Will select, reshape and normalize the correct parts of the variable buffer.

Parameters

var_bufferndarray

Array with the same size as the model’s variables array as returned by get_variables().

predict(X: numpy.ndarray)

Predict function

The decision is made for the label of the prototype with the minimum decision value, as provided by the decision_function().

Parameters

Xndarray

The data.

Returns

ndarray of shape (n_observations)

Returns the predicted labels.

predict_proba(X: numpy.ndarray)

Parameters

Xndarray

The data.

Returns

confidence_scoresndarray of shape (n_observations, n_classes)

score(X, y, sample_weight=None)

Return the mean accuracy on the given test data and labels.

In multi-label classification, this is the subset accuracy which is a harsh metric since you require for each sample that each label set be correctly predicted.

Parameters

Xarray-like of shape (n_samples, n_features)

Test samples.

yarray-like of shape (n_samples,) or (n_samples, n_outputs)

True labels for X.

sample_weightarray-like of shape (n_samples,), default=None

Sample weights.

Returns

scorefloat

Mean accuracy of self.predict(X) wrt. y.

set_model_params(new_model_params: Tuple[numpy.ndarray, numpy.ndarray])

Changes the model’s internal parameters. Copies the values of model_params into self.prototypes_ and self.omega_ therefore updating the self.variables_ array.

Parameters

new_model_paramstuple of ndarrays

Shapes depend on initialization but in the case of a square relevance matrix: tuple((n_prototypes, n_features), (n_matrices, n_features, n_features))

set_omega(omega)

Convenience function that makes sure to copy the value to self.omega_ and not overwrite it.

Parameters

omegandarray with same shape as self.omega_

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters

**paramsdict

Estimator parameters.

Returns

selfestimator instance

Estimator instance.

set_prototypes(new_prototypes: numpy.ndarray) → None

Accepts a new_prototypes array with the same shape as self.prototypes_ and overwrites the self._variables array by copying the values of the new_prototypes.

Parameters

new_prototypesndarray of shape (n_prototypes, n_features)

The new prototypes the model should store.

set_variables(new_variables: numpy.ndarray) → None

Modifies the self._variables by copying the values of new_variables into the memory of self._variables.

Parameters

new_variablesndarray

1d numpy array that contains all the model parameters in continuous memory

to_model_params_view(var_buffer: numpy.ndarray) → Tuple[numpy.ndarray, numpy.ndarray]

Parameters

var_bufferndarray

Array with the same size as the model’s variables array as returned by get_variables().

Returns

tuple

Returns a tuple with the prototypes and omega matrices as ndarrays.

to_omega(var_buffer: numpy.ndarray) → numpy.ndarray

Returns a view (of the shape of the model’s omega) into the provided variables buffer of the same size as the model’s variables array.

Parameters

var_bufferndarray

Array with the same size as the model’s variables array as returned by get_variables().

Returns

ndarray

Shape depending on initialization but in case of a square matrix (n_matrices, n_features, n_features).

to_prototypes_view(var_buffer: numpy.ndarray) → numpy.ndarray

Returns a view (of the shape of the model’s prototypes) into the provided variables buffer of the same size as the model’s variables array.

Parameters

var_bufferndarray

Array with the same size as the model’s variables array as returned by get_variables().

Returns

ndarray of shape (n_prototypes, n_features)

Prototype view into the var_buffer.

transform(X: numpy.ndarray, scale: bool = False, omega_hat_index: Union[int, List[int]] = 0) → numpy.ndarray

Parameters

Xnp.ndarray with shape (n_samples, n_features)

Data that needs to be transformed

scale{True, False}, default = False

Controls if the eigenvectors the data is projected on are scaled by the square root of their eigenvalues.

omega_hat_indexint or list

The indices of the omega_hats_ the transformation should be computed for.

Returns

The data projected on columns of omega_hat_ with shape (n_samples,

n_columns, n_matrices)