Ridge regressor with a kernel function. More...

#include <KernelRidgeRegressor.h>

Inheritance diagram for lucid::KernelRidgeRegressor:

Public Member Functions
	KernelRidgeRegressor (const Kernel &kernel, Scalar regularization_constant=0, const std::shared_ptr< const Tuner > &tuner=nullptr)
	Construct a new Kernel Ridge Regressor object with the given parameters.

	KernelRidgeRegressor (std::unique_ptr< Kernel > &&kernel, Scalar regularization_constant=0, const std::shared_ptr< const Tuner > &tuner=nullptr)
	Construct a new Kernel Ridge Regressor object with the given parameters.

Matrix	predict (ConstMatrixRef x) const override
	A model is a function that takes a \( n \times d_x \) matrix of row vectors in the input space \( \mathcal{X} \) and returns a \( n \times d_y \) matrix of row vectors in the output space \( \mathcal{Y} \).

Matrix	operator() (ConstMatrixRef x, const FeatureMap &feature_map) const
	A model is a function that takes a \( n \times d_x \) matrix of row vectors in the input space \( \mathcal{X} \) and returns a \( n \times d_y \) matrix of row vectors in the output space \( \mathcal{Y} \).

Matrix	predict (ConstMatrixRef x, const FeatureMap &feature_map) const
	A model is a function that takes a \( n \times d_x \) matrix of row vectors in the input space \( \mathcal{X} \) and returns a \( n \times d_y \) matrix of row vectors in the output space \( \mathcal{Y} \).

const std::unique_ptr< Kernel > &	kernel () const
	Get read-only access to the kernel of the regressor.

const Matrix &	training_inputs () const
	Get read-only access to the training inputs of the regressor.

const Matrix &	coefficients () const
	Get read-only access to the coefficients of the regressor.

double	regularization_constant () const
	Get read-only access to the regularization constant of the regressor.

double	log_marginal_likelihood () const
	Get read-only access to the log marginal likelihood of the regressor.

void	set (Parameter parameter, int value) override
	Set the `parameter` to the indicated `value`.

void	set (Parameter parameter, double value) override
	Set the `parameter` to the indicated `value`.

void	set (Parameter parameter, const Vector &value) override
	Set the `parameter` to the indicated `value`.

Estimator &	consolidate_impl (ConstMatrixRef training_inputs, ConstMatrixRef training_outputs, Requests requests) override
	Concrete implementation of the consolidate method.

double	score (ConstMatrixRef evaluation_inputs, ConstMatrixRef evaluation_outputs) const override
	Score the estimator assigning a numerical value to its accuracy in predicting the `evaluation_outputs` given the `evaluation_inputs`.

std::unique_ptr< Estimator >	clone () const override
	Clone the estimator by creating a new instance with the same parameters.

std::string	to_string () const override
	Obtain the string representation of this object.

Matrix	operator() (ConstMatrixRef x) const
	A model is a function that takes a \( n \times d_x \) matrix of row vectors in the input space \( \mathcal{X} \) and returns a \( n \times d_y \) matrix of row vectors in the output space \( \mathcal{Y} \).

Estimator &	consolidate (ConstMatrixRef training_inputs, ConstMatrixRef training_outputs, const Request request=Request::_)
	Consolidate the model, making sure it is ready for use.

Estimator &	consolidate (ConstMatrixRef training_inputs, ConstMatrixRef training_outputs, Requests requests)
	Consolidate the model, making sure it is ready for use.

Estimator &	fit (ConstMatrixRef training_inputs, ConstMatrixRef training_outputs, const Tuner &tuner)
	Fit the model to the given data.

Estimator &	fit (ConstMatrixRef training_inputs, ConstMatrixRef training_outputs)
	Fit the model to the given data.

Public Member Functions inherited from lucid::GradientOptimizable
double	objective_value () const
	Get read-only access to the objective value of the this object.

const Vector &	gradient () const
	Get read-only access to the gradient of the this object.

Public Member Functions inherited from lucid::Estimator
	Estimator (Parameters parameters=NoParameters, const std::shared_ptr< const Tuner > &tuner=nullptr)
	Construct a new Estimator object.

Matrix	operator() (ConstMatrixRef x) const
	A model is a function that takes a \( n \times d_x \) matrix of row vectors in the input space \( \mathcal{X} \) and returns a \( n \times d_y \) matrix of row vectors in the output space \( \mathcal{Y} \).

Estimator &	fit (ConstMatrixRef training_inputs, ConstMatrixRef training_outputs, const Tuner &tuner)
	Fit the model to the given data.

Estimator &	fit (ConstMatrixRef training_inputs, ConstMatrixRef training_outputs)
	Fit the model to the given data.

Estimator &	fit_online (ConstMatrixRef training_inputs, const OutputComputer &training_outputs, const Tuner &tuner)
	Fit the model to the data produced by the `training_outputs` OutputComputer.

Estimator &	fit_online (ConstMatrixRef training_inputs, const OutputComputer &training_outputs)
	Fit the model to the data produced by the `training_outputs` OutputComputer.

Estimator &	consolidate (ConstMatrixRef training_inputs, ConstMatrixRef training_outputs, const Request request=Request::_)
	Consolidate the model, making sure it is ready for use.

Estimator &	consolidate (ConstMatrixRef training_inputs, ConstMatrixRef training_outputs, Requests requests)
	Consolidate the model, making sure it is ready for use.

const std::shared_ptr< const Tuner > &	tuner () const
	Get read-only access to the tuner of the estimator.

std::shared_ptr< const Tuner > &	m_tuner ()
	Get read-write access to the tuner of the estimator.

Public Member Functions inherited from lucid::Parametrizable
template<IsAnyOf< int, double, const Vector & > T>
T	get (Parameter parameter) const
	Get the value of the specified `parameter`.

template<Parameter P>
internal::ParameterType< P >::ref_type	get () const
	Get the value of the specified `parameter`.

std::variant< int, double, Vector >	get (Parameter parameter) const
	Get the value of the specified `parameter`.

void	set (Parameter parameter, const std::variant< int, double, Vector > &value)
	Set the `parameter` to the indicated `value`.

template<Parameter P>
void	set (const std::variant< int, double, Vector > &value)
	Set the `parameter` to the indicated `value`.

void	set (Parameter parameter, std::size_t idx, const std::variant< std::vector< int >, std::vector< double >, std::vector< Vector > > &values)
	Set the `parameter` to the `index`-th value among the indicated `values`.

template<Parameter P>
void	set (std::size_t idx, const std::variant< std::vector< int >, std::vector< double >, std::vector< Vector > > &values)
	Set the `parameter` to the `index`-th value among the indicated `values`.

template<Parameter P>
void	set (typename internal::ParameterType< P >::ref_type value)
	Set the `parameter` to the indicated `value`.

bool	has (const Parameter parameter) const
	Check whether the `parameter` is present in this object.

Parameters	parameters () const
	Get read-only access to the parameters of the parametrizable object.

std::vector< Parameter >	parameters_list () const
	Get read-only access to the list of parameters of the parametrizable object.

Parametrizable &	load (const Parametrizable &o)
	Load parameters from another `Parametrizable` object.

Private Member Functions
int	get_i (Parameter parameter) const override
	Get the value of the specified `parameter`.

double	get_d (Parameter parameter) const override
	Get the value of the specified `parameter`.

const Vector &	get_v (Parameter parameter) const override
	Get the value of the specified `parameter`.

Private Attributes
std::unique_ptr< Kernel >	kernel_
	Kernel function.

double	regularization_constant_
	Regularization constant.

Matrix	training_inputs_
	Training inputs.

Matrix	coefficients_
	Coefficients of the linear combination describing the regression model.

Additional Inherited Members
Protected Attributes inherited from lucid::GradientOptimizable
double	objective_value_
	Objective value of the optimisation problem.

Vector	gradient_
	Gradient of the objective function.

Protected Attributes inherited from lucid::Parametrizable
Parameters	parameters_
	Parameters supported by this object.

Detailed Description

Ridge regressor with a kernel function.

This is a linear regressor with \( L_2 \) regularization. Given two vector spaces \( \mathcal{X} \subseteq \mathbb{R}^{d_x}, \mathcal{Y} \subseteq \mathbb{R}^{d_y} \) and the training datasets \( \{ (x_i, y_i) \}_{i=1}^n \), where \( x_i \in \mathcal{X} \) and \( y_i \in \mathcal{Y} \), the goal is to find the function \( f^*: \mathcal{X} \to \mathcal{Y} \) such that the sum of the squared errors is minimized, i.e.

\[ f^* = \arg\min_{f \in \mathcal{H}} \left\{ \frac{1}{n} \sum_{i=1}^n (y_i - f(x_i))^2 + \lambda \|f\|_{\mathcal{H}}^2 \right\} \]

where \( \mathcal{H} \) is a reproducing kernel Hilbert space (RKHS) with kernel \( k: \mathcal{X} \times \mathcal{X} \to \mathbb{R} \). Due to the reproducing property of the RKHS, \( f^* \) can be expressed as the linear combination

\[ f^*(x) = \sum_{i=1}^n w_i k(x_i, x) \]

where \( w \in \mathbb{R}^n \) are some coefficients we want to find. We can rewrite the loss function in vector form as

\[\frac{1}{n} \|y - Kw\|^2 + \lambda w^T K w \]

where \( K \) is the Gram matrix with elements \( K_{ij} = k(x_i, x_j) \). Looking for the minimum, the partial gradient of the loss function with respect to \( w \) is set to zero

\[\frac{\partial}{\partial w} \left\{ \frac{1}{n} \|y - Kw\|^2 + \lambda w^T K w \right\} = 0 \]

leads us to the closed-form solution:

\[w = (K + \lambda n I)^{-1} y . \]

Finally, when we want to predict the output for a new input \( x \), we can just compute

\[f^*(x) = k(x, x_1) w_1 + k(x, x_2) w_2 + \cdots + k(x, x_n) w_n \]

of, in matrix form

\[f^*(x) = K(x, x_\text{traning}) w \]

where \( K(x, x_\text{traning}) \) is the vector of kernel evaluations between \( x \) and the training inputs \( x_\text{traning} \).

Constructor & Destructor Documentation

◆ KernelRidgeRegressor() [1/2]

lucid::KernelRidgeRegressor::KernelRidgeRegressor	(	const Kernel &	kernel,
		Scalar	regularization_constant = 0,
		const std::shared_ptr< const Tuner > &	tuner = nullptr )

explicit

Construct a new Kernel Ridge Regressor object with the given parameters.

Parameters

kernel	kernel function used to compute the Gram matrix
regularization_constant	regularization constant. Avoids overfitting by penalizing large coefficients
tuner	tuner used to fit the model

◆ KernelRidgeRegressor() [2/2]

lucid::KernelRidgeRegressor::KernelRidgeRegressor	(	std::unique_ptr< Kernel > &&	kernel,
		Scalar	regularization_constant = 0,
		const std::shared_ptr< const Tuner > &	tuner = nullptr )

explicit

Construct a new Kernel Ridge Regressor object with the given parameters.

Parameters

kernel	kernel function used to compute the Gram matrix
regularization_constant	regularization constant. Avoids overfitting by penalizing large coefficients
tuner	tuner used to fit the model

Member Function Documentation

◆ clone()

std::unique_ptr< Estimator > lucid::KernelRidgeRegressor::clone ( ) const

nodiscardoverridevirtual

Clone the estimator by creating a new instance with the same parameters.

Returns: new instance of the estimator

Implements lucid::Estimator.

◆ coefficients()

const Matrix & lucid::KernelRidgeRegressor::coefficients ( ) const

inlinenodiscard

Get read-only access to the coefficients of the regressor.

Returns: coefficients of the regressor

◆ consolidate() [1/2]

Estimator & lucid::Estimator::consolidate	(	ConstMatrixRef	training_inputs,
		ConstMatrixRef	training_outputs,
		const Request	request = Request::_ )

inline

Consolidate the model, making sure it is ready for use.

No fitting process is performed, and the hyperparameters are not updated, but the estimator may change its internal state so it can be used for predictions. After the process is completed, the estimator can be used to make predictions on new data. The request parameter is used to specify an additional option for the consolidation process, like whether to compute some additional values used by some tuners.

Note: This is a low-level method that allows the user great control over the behaviour of the estimator. It is usually recommended to use fit with the desired tuner instead.

Precondition: The number of rows in the training inputs should be equal to the number of rows in the training outputs.

Parameters

training_inputs	training input data. The number of rows should be equal to the number of training outputs
training_outputs	training output data. The number of rows should be equal to the number of training inputs
request	request for the consolidation process. Defaults to no specific request being made

Returns: reference to the estimator

◆ consolidate() [2/2]

Estimator & lucid::Estimator::consolidate	(	ConstMatrixRef	training_inputs,
		ConstMatrixRef	training_outputs,
		Requests	requests )

Consolidate the model, making sure it is ready for use.

No fitting process is performed, and the hyperparameters are not updated, but the estimator may change its internal state so it can be used for predictions. After the process is completed, the estimator can be used to make predictions on new data. The requests parameter can be used to specify additional options for the consolidation process, like whether to compute some additional values used by some tuners.

Note: This is a low-level method that allows the user great control over the behaviour of the estimator. It is usually recommended to use fit with the desired tuner instead.

Precondition: The number of rows in the training inputs should be equal to the number of rows in the training outputs.

Parameters

training_inputs	training input data
training_outputs	training output data
requests	requests for the consolidation process

Returns: reference to the estimator

◆ consolidate_impl()

Estimator & lucid::KernelRidgeRegressor::consolidate_impl	(	ConstMatrixRef	training_inputs,
		ConstMatrixRef	training_outputs,
		Requests	requests )

overridevirtual

Concrete implementation of the consolidate method.

Parameters

training_inputs	training input data
training_outputs	training output data
requests	requests for the consolidation process

Returns: reference to the estimator

Implements lucid::Estimator.

◆ fit() [1/2]

Estimator & lucid::Estimator::fit	(	ConstMatrixRef	training_inputs,
		ConstMatrixRef	training_outputs )

Fit the model to the given data.

This method will use the object's tuner to find the best hyperparameters for the model. After the process is completed, the estimator can be used to make predictions on new data. If no tuner has been provided during construction, the method is equivalent to consolidate with no specific request.

Precondition: The number of rows in the training inputs should be equal to the number of rows in the training outputs.

Parameters

training_inputs	training input data
training_outputs	training output data

Returns: reference to this estimator after it has been fitted

◆ fit() [2/2]

Estimator & lucid::Estimator::fit	(	ConstMatrixRef	training_inputs,
		ConstMatrixRef	training_outputs,
		const Tuner &	tuner )

Fit the model to the given data.

This method will use the provided tuner to find the best hyperparameters for the model. After the process is completed, the estimator can be used to make predictions on new data.

Precondition: The number of rows in the training inputs should be equal to the number of rows in the training outputs.

Parameters

training_inputs	training input data
training_outputs	training output data
tuner	Tuner object used to find the best hyperparameters for the model

Returns: reference to this estimator after it has been fitted

◆ get_d()

double lucid::KernelRidgeRegressor::get_d ( Parameter parameter ) const

nodiscardoverrideprivatevirtual

Get the value of the specified parameter.

Parameters

parameter parameter to retrieve

Returns: value of the parameter

Precondition: The parameter must be present and be associated with a value of type double.

Reimplemented from lucid::Parametrizable.

◆ get_i()

int lucid::KernelRidgeRegressor::get_i ( Parameter parameter ) const

nodiscardoverrideprivatevirtual

Get the value of the specified parameter.

Parameters

parameter parameter to retrieve

Returns: value of the parameter

Precondition: The parameter must be present and be associated with a value of type int.

Reimplemented from lucid::Parametrizable.

◆ get_v()

const Vector & lucid::KernelRidgeRegressor::get_v ( Parameter parameter ) const

nodiscardoverrideprivatevirtual

Get the value of the specified parameter.

Parameters

parameter parameter to retrieve

Returns: value of the parameter

Precondition: The parameter must be present and be associated with a value of type Vector.

Reimplemented from lucid::Parametrizable.

◆ kernel()

const std::unique_ptr< Kernel > & lucid::KernelRidgeRegressor::kernel ( ) const

inlinenodiscard

Get read-only access to the kernel of the regressor.

Returns: kernel of the regressor

◆ log_marginal_likelihood()

double lucid::KernelRidgeRegressor::log_marginal_likelihood ( ) const

inlinenodiscard

Get read-only access to the log marginal likelihood of the regressor.

Returns: log marginal likelihood of the regressor

◆ operator()() [1/2]

Matrix lucid::Estimator::operator() ( ConstMatrixRef x ) const

nodiscard

A model is a function that takes a \( n \times d_x \) matrix of row vectors in the input space \( \mathcal{X} \) and returns a \( n \times d_y \) matrix of row vectors in the output space \( \mathcal{Y} \).

Precondition: The estimator should be fitted before calling this method.

Parameters

x	\( n \times d_x \) matrix of row vectors in \( \mathcal{X} \)

Returns: \( n \times d_y \) matrix of row vectors in \( \mathcal{Y} \)

◆ operator()() [2/2]

Matrix lucid::KernelRidgeRegressor::operator()	(	ConstMatrixRef	x,
		const FeatureMap &	feature_map ) const

nodiscard

A model is a function that takes a \( n \times d_x \) matrix of row vectors in the input space \( \mathcal{X} \) and returns a \( n \times d_y \) matrix of row vectors in the output space \( \mathcal{Y} \).

The feature_map is used to approximate the kernel vector \( k(x, x_i) \).

Warning: Using this method introduces an approximation error. We suggest using the operator()(ConstMatrixRef) method instead.

Parameters

x	\( n \times d_x \) matrix of row vectors in \( \mathcal{X} \)
feature_map	feature map used to approximate the kernel vector \( k(x, x_i) \)

Returns: \( n \times d_y \) matrix of row vectors in \( \mathcal{Y} \)

◆ predict() [1/2]

Matrix lucid::KernelRidgeRegressor::predict ( ConstMatrixRef x ) const

nodiscardoverridevirtual

A model is a function that takes a \( n \times d_x \) matrix of row vectors in the input space \( \mathcal{X} \) and returns a \( n \times d_y \) matrix of row vectors in the output space \( \mathcal{Y} \).

Precondition: The estimator should be fitted before calling this method.

Parameters

x	\( n \times d_x \) matrix of row vectors in \( \mathcal{X} \)

Returns: \( n \times d_y \) matrix of row vectors in \( \mathcal{Y} \)

Implements lucid::Estimator.

◆ predict() [2/2]

Matrix lucid::KernelRidgeRegressor::predict	(	ConstMatrixRef	x,
		const FeatureMap &	feature_map ) const

nodiscard

A model is a function that takes a \( n \times d_x \) matrix of row vectors in the input space \( \mathcal{X} \) and returns a \( n \times d_y \) matrix of row vectors in the output space \( \mathcal{Y} \).

The feature_map is used to approximate the kernel vector \( k(x, x_i) \).

Warning: Using this method introduces an approximation error. We suggest using the operator()(ConstMatrixRef) method instead.

Parameters

x	\( n \times d_x \) matrix of row vectors in \( \mathcal{X} \)
feature_map	feature map used to approximate the kernel vector \( k(x, x_i) \)

Returns: \( n \times d_y \) matrix of row vectors in \( \mathcal{Y} \)

◆ regularization_constant()

double lucid::KernelRidgeRegressor::regularization_constant ( ) const

inlinenodiscard

Get read-only access to the regularization constant of the regressor.

Returns: regularization constant of the regressor

◆ score()

double lucid::KernelRidgeRegressor::score	(	ConstMatrixRef	evaluation_inputs,
		ConstMatrixRef	evaluation_outputs ) const

nodiscardoverridevirtual

Score the estimator assigning a numerical value to its accuracy in predicting the evaluation_outputs given the evaluation_inputs.

Given the evaluation inputs \( x = \{ x_1, \dots, x_n \} \), where \( x_i \in \mathcal{X} \subseteq \mathbb{R}^{d_x}, 0 \le i \le n \), we want to give a numerical score to the model's predictions \( \hat{y} = \{ \hat{y}_1, \dots, \hat{y}_n \} \) with respect to the true outputs \( y = \{ y_1, \dots, y_n \} \), where \( y_i \in \mathcal{Y} \subseteq \mathbb{R}^{d_y}, 0 \le i \le n \).

Note: By default, the score is computed using the scorer::r2_score.

Precondition: The methods fit or consolidate should have been called at least once before calling this method.

Parameters

evaluation_inputs	\( \texttip{n}{Number of samples} \times \texttip{d_x}{Dimension of the input vector space} \) evaluation input data
evaluation_outputs	\( \texttip{n}{Number of samples} \times \texttip{d_y}{Dimension of the output vector space} \) evaluation output data

Returns: score of the model

Implements lucid::Estimator.

◆ set() [1/3]

void lucid::KernelRidgeRegressor::set	(	Parameter	parameter,
		const Vector &	value )

overridevirtual

Set the parameter to the indicated value.

Parameters

parameter	parameter to set
value	value to assign to the specified parameter

Precondition: The parameter must be present and be associated with a value of type Vector.

Reimplemented from lucid::Parametrizable.

◆ set() [2/3]

void lucid::KernelRidgeRegressor::set	(	Parameter	parameter,
		double	value )

overridevirtual

Set the parameter to the indicated value.

Parameters

parameter	parameter to set
value	value to assign to the specified parameter

Precondition: The parameter must be present and be associated with a value of type double.

Reimplemented from lucid::Parametrizable.

◆ set() [3/3]

void lucid::KernelRidgeRegressor::set	(	Parameter	parameter,
		int	value )

overridevirtual

Set the parameter to the indicated value.

Parameters

parameter	parameter to set
value	value to assign to the specified parameter

Precondition: The parameter must be present and be associated with a value of type int.

Reimplemented from lucid::Parametrizable.

◆ to_string()

std::string lucid::KernelRidgeRegressor::to_string ( ) const

nodiscardoverridevirtual

Obtain the string representation of this object.

Returns: string representation of this object

Reimplemented from lucid::Estimator.

◆ training_inputs()

const Matrix & lucid::KernelRidgeRegressor::training_inputs ( ) const

inlinenodiscard

Get read-only access to the training inputs of the regressor.

Returns: training inputs of the regressor

The documentation for this class was generated from the following files:

lucid/model/KernelRidgeRegressor.h
lucid/model/KernelRidgeRegressor.cpp

Public Member Functions

Private Member Functions

Private Attributes

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ KernelRidgeRegressor() [1/2]

◆ KernelRidgeRegressor() [2/2]

Member Function Documentation

◆ clone()

◆ coefficients()

◆ consolidate() [1/2]

◆ consolidate() [2/2]

◆ consolidate_impl()

◆ fit() [1/2]

◆ fit() [2/2]

◆ get_d()

◆ get_i()

◆ get_v()

◆ kernel()

◆ log_marginal_likelihood()

◆ operator()() [1/2]

◆ operator()() [2/2]

◆ predict() [1/2]

◆ predict() [2/2]

◆ regularization_constant()

◆ score()

◆ set() [1/3]

◆ set() [2/3]

◆ set() [3/3]

◆ to_string()

◆ training_inputs()