Table of Contents

Class KernelRidgeRegression<T>

Namespace
AiDotNet.Regression
Assembly
AiDotNet.dll

Implements Kernel Ridge Regression, a powerful nonlinear regression technique that combines ridge regression with the kernel trick to capture complex nonlinear relationships.

public class KernelRidgeRegression<T> : NonLinearRegressionBase<T>, INonLinearRegression<T>, IRegression<T>, IFullModel<T, Matrix<T>, Vector<T>>, IModel<Matrix<T>, Vector<T>, ModelMetadata<T>>, IModelSerializer, ICheckpointableModel, IParameterizable<T, Matrix<T>, Vector<T>>, IFeatureAware, IFeatureImportance<T>, ICloneable<IFullModel<T, Matrix<T>, Vector<T>>>, IGradientComputable<T, Matrix<T>, Vector<T>>, IJitCompilable<T>

Type Parameters

T

The numeric type used for calculations, typically float or double.

Inheritance
KernelRidgeRegression<T>
Implements
IFullModel<T, Matrix<T>, Vector<T>>
IModel<Matrix<T>, Vector<T>, ModelMetadata<T>>
IParameterizable<T, Matrix<T>, Vector<T>>
ICloneable<IFullModel<T, Matrix<T>, Vector<T>>>
IGradientComputable<T, Matrix<T>, Vector<T>>
Inherited Members
Extension Methods

Remarks

Kernel Ridge Regression extends linear ridge regression by applying the "kernel trick" to implicitly map the input features to a higher-dimensional space without explicitly computing the transformation. This allows the model to capture complex nonlinear relationships while still maintaining the computational efficiency of ridge regression. The regularization parameter (lambda) helps prevent overfitting by penalizing large coefficients.

For Beginners: Kernel Ridge Regression is like using a special lens that helps see complex patterns in your data.

Regular linear models can only fit straight lines to data, but many real-world relationships aren't straight. Kernel Ridge Regression solves this by:

  • Using "kernels" to transform your data into a format where complex relationships become simpler
  • Finding patterns in this transformed space
  • Adding "ridge" regularization to prevent the model from becoming too complex or overfitting

Think of it like this: If you tried to separate red and blue dots on a sheet of paper with a single line, sometimes it's impossible. But if you could lift some dots off the page (into 3D space), you might be able to separate them with a flat plane. Kernels do something similar - they transform your data so that complex patterns become easier to find.

This technique is particularly good for:

  • Medium-sized datasets with complex relationships
  • Problems where the relationship between inputs and outputs is highly nonlinear
  • When you need both good prediction accuracy and the ability to adjust how much the model fits to noise

Constructors

KernelRidgeRegression(KernelRidgeRegressionOptions, IRegularization<T, Matrix<T>, Vector<T>>?)

Initializes a new instance of the KernelRidgeRegression<T> class.

public KernelRidgeRegression(KernelRidgeRegressionOptions options, IRegularization<T, Matrix<T>, Vector<T>>? regularization = null)

Parameters

options KernelRidgeRegressionOptions

Configuration options for the Kernel Ridge Regression algorithm.

regularization IRegularization<T, Matrix<T>, Vector<T>>

Optional regularization strategy to prevent overfitting.

Remarks

This constructor creates a new Kernel Ridge Regression model with the specified options and regularization strategy. The options control parameters such as the regularization strength (lambda) and the matrix decomposition type. If no regularization is specified, no regularization is applied beyond the ridge penalty.

For Beginners: This is how you create a new Kernel Ridge Regression model.

When creating a model, you need to specify:

  • Options: Controls settings like the regularization strength (lambda), which determines how much the model balances between fitting the training data perfectly and keeping the model simple
  • Regularization: Optional additional method to prevent overfitting

Example:

// Create options for Kernel Ridge Regression
var options = new KernelRidgeRegressionOptions { 
    LambdaKRR = 0.1,
    KernelType = KernelType.RBF
};

// Create the model
var krr = new KernelRidgeRegression<double>(options);

Methods

CreateInstance()

Creates a new instance of the KernelRidgeRegression with the same configuration as the current instance.

protected override IFullModel<T, Matrix<T>, Vector<T>> CreateInstance()

Returns

IFullModel<T, Matrix<T>, Vector<T>>

A new KernelRidgeRegression instance with the same options and regularization as the current instance.

Remarks

This method creates a new instance of the KernelRidgeRegression model with the same configuration options and regularization settings as the current instance. This is useful for model cloning, ensemble methods, or cross-validation scenarios where multiple instances of the same model with identical configurations are needed.

For Beginners: This method creates a fresh copy of the model's blueprint.

When you need multiple versions of the same type of model with identical settings:

  • This method creates a new, empty model with the same configuration
  • It's like making a copy of a recipe before you start cooking
  • The new model has the same settings but no trained data
  • This is useful for techniques that need multiple models, like cross-validation

For example, when testing your model on different subsets of data, you'd want each test to use a model with identical settings.

Deserialize(byte[])

Loads a previously serialized Kernel Ridge Regression model from a byte array.

public override void Deserialize(byte[] modelData)

Parameters

modelData byte[]

The byte array containing the serialized model.

Remarks

This method reconstructs a Kernel Ridge Regression model from a byte array that was previously created using the Serialize method. It restores the base class data, model-specific options, the Gram matrix, and the dual coefficients, allowing the model to be used for predictions without retraining.

For Beginners: This method loads a previously saved model from a sequence of bytes.

Deserialization allows you to:

  • Load a model that was saved earlier
  • Use a model without having to retrain it
  • Share models between different applications

When you deserialize a model:

  • All settings are restored
  • The Gram matrix is reconstructed
  • The dual coefficients are recovered
  • The model is ready to make predictions immediately

Example:

// Load from a file
byte[] modelData = File.ReadAllBytes("kernelRidgeRegression.model");

// Deserialize the model
var options = new KernelRidgeRegressionOptions();
var krr = new KernelRidgeRegression<double>(options);
krr.Deserialize(modelData);

// Now you can use the model for predictions
var predictions = krr.Predict(newFeatures);

GetModelMetadata()

Gets metadata about the Kernel Ridge Regression model and its configuration.

public override ModelMetadata<T> GetModelMetadata()

Returns

ModelMetadata<T>

A ModelMetadata object containing information about the model.

Remarks

This method returns metadata about the model, including its type, the regularization parameter (lambda), and the regularization type. This information can be useful for model management, comparison, and documentation purposes.

For Beginners: This method provides information about your model configuration.

The metadata includes:

  • The type of model (Kernel Ridge Regression)
  • The lambda value (regularization strength)
  • The type of regularization applied

This information is helpful when:

  • Comparing different models
  • Documenting your model's configuration
  • Troubleshooting model performance

Example:

var metadata = krr.GetModelMetadata();
Console.WriteLine($"Model type: {metadata.ModelType}");
Console.WriteLine($"Lambda: {metadata.AdditionalInfo["LambdaKRR"]}");

GetModelType()

Gets the model type of the Kernel Ridge Regression model.

protected override ModelType GetModelType()

Returns

ModelType

The model type enumeration value.

OptimizeModel(Matrix<T>, Vector<T>)

Optimizes the Kernel Ridge Regression model based on the provided training data.

protected override void OptimizeModel(Matrix<T> X, Vector<T> y)

Parameters

X Matrix<T>

A matrix where each row represents a sample and each column represents a feature.

y Vector<T>

A vector of target values corresponding to each sample in X.

Remarks

This method builds the Kernel Ridge Regression model by computing the Gram (kernel) matrix, which represents the similarity between all pairs of training samples, adding a ridge penalty to the diagonal for regularization, and solving for the dual coefficients. These coefficients, along with the support vectors (training samples), are used for making predictions.

For Beginners: This method does the actual "learning" for the model.

Here's what happens during this process:

  1. The model calculates how similar each training example is to every other training example (this creates what's called the "Gram matrix" or "kernel matrix")
  2. It adds the ridge penalty to the diagonal of this matrix (this is the regularization that helps prevent overfitting)
  3. It solves a mathematical equation to find the best "dual coefficients" - these determine how much each training example influences predictions
  4. It stores these coefficients and the training data (support vectors) for making predictions later

This process allows the model to capture complex patterns in your data while still maintaining good generalization to new, unseen data.

PredictSingle(Vector<T>)

Predicts the target value for a single input feature vector.

protected override T PredictSingle(Vector<T> input)

Parameters

input Vector<T>

The feature vector of the sample to predict.

Returns

T

The predicted value for the input sample.

Remarks

This method predicts the target value for a single input feature vector by computing the kernel function between the input and all support vectors (training samples), weighting the results by the dual coefficients, and summing them. This is the dual form of the prediction equation for kernel-based methods.

For Beginners: This method makes a prediction for a new data point.

To make a prediction:

  1. The model compares the new input to every training example (support vector) using the kernel function to measure similarity
  2. It multiplies each similarity value by the corresponding dual coefficient (which was learned during training)
  3. It adds up all these weighted similarity values to get the final prediction

This approach allows the model to make predictions based on how similar the new input is to the training examples, with more influential examples (those with larger dual coefficients) having a greater impact on the prediction.

Serialize()

Serializes the Kernel Ridge Regression model to a byte array for storage or transmission.

public override byte[] Serialize()

Returns

byte[]

A byte array containing the serialized model.

Remarks

This method converts the Kernel Ridge Regression model into a byte array that can be stored in a file, database, or transmitted over a network. The serialized data includes the base class data, model-specific options like the regularization parameter (lambda), the Gram matrix, and the dual coefficients.

For Beginners: This method saves your trained model as a sequence of bytes.

Serialization allows you to:

  • Save your model to a file
  • Store your model in a database
  • Send your model over a network
  • Keep your model for later use without having to retrain it

The serialized data includes:

  • The model's settings (like the lambda regularization parameter)
  • The Gram matrix (similarities between training examples)
  • The dual coefficients learned during training

Example:

// Serialize the model
byte[] modelData = krr.Serialize();

// Save to a file
File.WriteAllBytes("kernelRidgeRegression.model", modelData);
Edit this page