Table of Contents

Class SupportVectorRegression<T>

Namespace
AiDotNet.Regression
Assembly
AiDotNet.dll

Implements Support Vector Regression (SVR), which creates a regression model by finding a hyperplane that lies within a specified margin (epsilon) of the training data. This approach is effective for both linear and nonlinear regression problems.

public class SupportVectorRegression<T> : NonLinearRegressionBase<T>, INonLinearRegression<T>, IRegression<T>, IFullModel<T, Matrix<T>, Vector<T>>, IModel<Matrix<T>, Vector<T>, ModelMetadata<T>>, IModelSerializer, ICheckpointableModel, IParameterizable<T, Matrix<T>, Vector<T>>, IFeatureAware, IFeatureImportance<T>, ICloneable<IFullModel<T, Matrix<T>, Vector<T>>>, IGradientComputable<T, Matrix<T>, Vector<T>>, IJitCompilable<T>

Type Parameters

T

The numeric type used for calculations (typically float or double).

Inheritance
SupportVectorRegression<T>
Implements
IFullModel<T, Matrix<T>, Vector<T>>
IModel<Matrix<T>, Vector<T>, ModelMetadata<T>>
IParameterizable<T, Matrix<T>, Vector<T>>
ICloneable<IFullModel<T, Matrix<T>, Vector<T>>>
IGradientComputable<T, Matrix<T>, Vector<T>>
Inherited Members
Extension Methods

Remarks

Support Vector Regression (SVR) works by: - Transforming data into a higher-dimensional space using kernel functions - Finding the optimal hyperplane that fits within an epsilon-width tube around the data - Using only a subset of the training examples (support vectors) to make predictions - Balancing model complexity and training error through the C parameter

Unlike traditional regression methods that minimize squared errors, SVR aims to find a function that deviates from training data by no more than epsilon while remaining as flat as possible.

For Beginners: Support Vector Regression is like creating a tunnel through your data.

Think of it like this:

  • You want to draw a line (or curve) through your data points
  • Instead of drawing the line directly through the points, you create a tunnel of a certain width
  • You try to include as many points as possible inside this tunnel
  • Points outside the tunnel are called "support vectors" and help define its shape

For example, when predicting house prices, SVR would create a tunnel through the data that captures the general trend while allowing some houses to fall outside the tunnel if they're unusually priced for their features.

Constructors

SupportVectorRegression(SupportVectorRegressionOptions?, IRegularization<T, Matrix<T>, Vector<T>>?)

Creates a new Support Vector Regression model.

public SupportVectorRegression(SupportVectorRegressionOptions? options = null, IRegularization<T, Matrix<T>, Vector<T>>? regularization = null)

Parameters

options SupportVectorRegressionOptions

Optional configuration settings for the SVR model. These settings control aspects like:

  • The width of the epsilon-insensitive tube (epsilon)
  • The regularization parameter (C)
  • The type of kernel function to use If not provided, default options will be used.
regularization IRegularization<T, Matrix<T>, Vector<T>>

Optional regularization method to prevent overfitting. If not provided, no additional regularization will be applied beyond the built-in regularization of SVR.

Remarks

This constructor creates a new Support Vector Regression model with the specified configuration options and regularization method. If options are not provided, default values are used.

For Beginners: This method sets up a new SVR model before training.

Think of it like configuring a new tool:

  • You can use the default settings (by not specifying options)
  • Or you can customize how it works (by providing specific options)
  • The regularization parameter provides extra protection against overfitting

After setting up the model with these options, you'll need to train it on your data to find the support vectors and coefficients that best describe your data pattern.

Methods

CreateInstance()

Creates a new instance of the Support Vector Regression model with the same configuration.

protected override IFullModel<T, Matrix<T>, Vector<T>> CreateInstance()

Returns

IFullModel<T, Matrix<T>, Vector<T>>

A new instance of the Support Vector Regression model.

Remarks

This method creates a deep copy of the current Support Vector Regression model, including its options, support vectors, alpha coefficients, bias term, and regularization settings. The new instance is completely independent of the original, allowing modifications without affecting the original model.

For Beginners: This method creates an exact copy of your trained model.

Think of it like making a perfect duplicate of your tunnel:

  • It copies all the configuration settings (like epsilon, C, and kernel type)
  • It preserves the support vectors (the key data points that define your tunnel)
  • It maintains the alpha coefficients (how important each support vector is)
  • It keeps the bias term (B) which affects the overall position of your tunnel

Creating a copy is useful when you want to:

  • Create a backup before further modifying the model
  • Create variations of the same model for different purposes
  • Share the model with others while keeping your original intact

Exceptions

InvalidOperationException

Thrown when the creation fails or required components are null.

Deserialize(byte[])

Deserializes the support vector regression model from a byte array.

public override void Deserialize(byte[] modelData)

Parameters

modelData byte[]

The byte array containing the serialized model data.

Remarks

This method reconstructs the model from a byte array created by the Serialize method. It restores the model's coefficients, support vectors, and configuration options, allowing a previously saved model to be loaded and used for predictions.

For Beginners: This method loads a saved model from computer memory.

Think of it like opening a saved document:

  • It takes the byte array created by the Serialize method
  • It rebuilds all the settings, support vectors, and coefficients
  • The model is then ready to use for making predictions

This allows you to:

  • Use a previously trained model without having to train it again
  • Load models that others have shared with you
  • Use the same model across different applications

GetModelMetadata()

Gets metadata about the SVR model.

public override ModelMetadata<T> GetModelMetadata()

Returns

ModelMetadata<T>

A ModelMetadata object containing information about the model.

Remarks

This method returns metadata about the SVR model, including: - Base metadata from the parent class - The epsilon parameter (width of the insensitive tube) - The C parameter (regularization strength) - The type of regularization used

This metadata is useful for model inspection, logging, and debugging.

For Beginners: This method shares details about your model's configuration.

It's like getting a summary sheet about your model:

  • It includes the basic information about your model
  • It adds SVR-specific settings like epsilon and C
  • It notes what type of regularization was used

This information is helpful for comparing different models or documenting which settings worked best for your problem.

GetModelType()

Returns the type identifier for this regression model.

protected override ModelType GetModelType()

Returns

ModelType

The model type identifier for support vector regression.

Remarks

This method returns the enum value that identifies this model as a support vector regression model. This is used for model identification in serialization/deserialization and for logging purposes.

For Beginners: This method simply tells the system what kind of model this is.

It's like a name tag for the model that says "I am a support vector regression model." This is useful when:

  • Saving the model to a file
  • Loading a model from a file
  • Logging information about the model

You generally won't need to call this method directly in your code.

OptimizeModel(Matrix<T>, Vector<T>)

Optimizes the SVR model using the provided input data and target values.

protected override void OptimizeModel(Matrix<T> x, Vector<T> y)

Parameters

x Matrix<T>

The input feature matrix, where rows represent observations and columns represent features.

y Vector<T>

The target values vector containing the actual output values to predict.

Remarks

This method implements the core optimization for SVR. It: 1. Applies regularization to the input matrix 2. Uses the Sequential Minimal Optimization (SMO) algorithm to find the optimal parameters 3. Applies regularization to the resulting alpha coefficients

The SMO algorithm is an efficient way to solve the quadratic programming problem that arises when training an SVR model.

For Beginners: This method finds the best tunnel through your data points.

The process works like this:

  1. The model prepares your data with regularization (like smoothing it out)
  2. It uses a special algorithm (SMO) to find the optimal tunnel shape
  3. It identifies which points (support vectors) are most important for defining this tunnel
  4. It calculates how much influence each support vector should have on predictions

After optimization, the model knows exactly how to predict new values based on the patterns it found in your training data.

Predict(Matrix<T>)

Predicts target values for a matrix of input features.

public override Vector<T> Predict(Matrix<T> input)

Parameters

input Matrix<T>

The input feature matrix for which to make predictions.

Returns

Vector<T>

A vector of predicted values, one for each row in the input matrix.

Remarks

This method makes predictions for multiple input samples. It predicts each sample individually using the PredictSingle method, computing kernel similarities with support vectors. Note that SVR regularization is controlled through the C parameter during training, not through data transformation during prediction.

For Beginners: This method uses your trained model to predict values for new data.

The prediction process works like this: For each row of data (like each house you want to price):

  • It calculates how similar this house is to each support vector
  • It combines these similarities using the trained weights (alphas)
  • It adds the bias term (B) to get the final prediction

This lets you predict many values at once, like estimating prices for multiple houses simultaneously.

PredictSingle(Vector<T>)

Predicts a target value for a single input feature vector.

protected override T PredictSingle(Vector<T> input)

Parameters

input Vector<T>

The input feature vector for which to make a prediction.

Returns

T

The predicted value for the input vector.

Remarks

This method implements the SVR prediction function for a single input sample: f(x) = b + sum(alpha_i * K(support_vector_i, x))

where:

  • b is the bias term (B)
  • alpha_i are the Lagrange multipliers (Alphas)
  • K is the kernel function
  • support_vector_i are the training examples that define the model

For Beginners: This method predicts a value for a single data point.

Think of it like this:

  1. Start with a base value (the bias term B)
  2. For each support vector (key data points from training):
    • Calculate how similar your input is to this support vector using the kernel function
    • Multiply this similarity by the importance (alpha) of that support vector
    • Add this contribution to your prediction
  3. The final sum is your predicted value

For example, predicting a house price might involve comparing the house to several key houses (support vectors) from your training data, with each comparison weighted by how important that house is for making predictions.

Serialize()

Serializes the support vector regression model to a byte array for storage or transmission.

public override byte[] Serialize()

Returns

byte[]

A byte array containing the serialized model data.

Remarks

This method converts the model, including its coefficients, support vectors, and configuration options, into a byte array. This enables the model to be saved to a file, stored in a database, or transmitted over a network.

For Beginners: This method saves the model to computer memory so you can use it later.

Think of it like taking a snapshot of the model:

  • It captures all the important values, settings, and support vectors
  • It converts them into a format that can be easily stored
  • The resulting byte array can be saved to a file or database

This is useful when you want to:

  • Train the model once and use it many times
  • Share the model with others
  • Use the model in a different application
Edit this page