Table of Contents

Class ElasticNetRegression<T>

Namespace
AiDotNet.Regression
Assembly
AiDotNet.dll

Implements Elastic Net Regression (combined L1 and L2 regularized linear regression), which extends ordinary least squares by adding both L1 (Lasso) and L2 (Ridge) penalty terms.

public class ElasticNetRegression<T> : RegressionBase<T>, IRegression<T>, IFullModel<T, Matrix<T>, Vector<T>>, IModel<Matrix<T>, Vector<T>, ModelMetadata<T>>, IModelSerializer, ICheckpointableModel, IParameterizable<T, Matrix<T>, Vector<T>>, IFeatureAware, IFeatureImportance<T>, ICloneable<IFullModel<T, Matrix<T>, Vector<T>>>, IGradientComputable<T, Matrix<T>, Vector<T>>, IJitCompilable<T>

Type Parameters

T

The numeric type used for calculations (typically float or double).

Inheritance
ElasticNetRegression<T>
Implements
IFullModel<T, Matrix<T>, Vector<T>>
IModel<Matrix<T>, Vector<T>, ModelMetadata<T>>
IParameterizable<T, Matrix<T>, Vector<T>>
ICloneable<IFullModel<T, Matrix<T>, Vector<T>>>
IGradientComputable<T, Matrix<T>, Vector<T>>
Inherited Members
Extension Methods

Remarks

Elastic Net Regression solves the following optimization problem: minimize (1/2n) * ||y - Xw||^2 + alpha * l1_ratio * ||w||_1 + alpha * (1 - l1_ratio) * ||w||^2 / 2

Like Lasso, Elastic Net uses coordinate descent optimization because the L1 penalty is not differentiable at zero.

Elastic Net combines the benefits of both Ridge and Lasso:

  • Feature selection from L1 (can set coefficients to exactly zero)
  • Stability with correlated features from L2 (groups correlated features together)

For Beginners: Elastic Net gives you the best of both worlds.

Lasso is great at selecting important features, but when features are correlated, it tends to arbitrarily pick one and zero out the others. Ridge handles correlated features well but doesn't do feature selection.

Elastic Net solves both problems:

  • It can still set coefficients to zero (like Lasso) for feature selection
  • It groups correlated features together (like Ridge) instead of picking arbitrarily

Example usage:

var options = new ElasticNetRegressionOptions<double> { Alpha = 1.0, L1Ratio = 0.5 };
var elasticNet = new ElasticNetRegression<double>(options);
elasticNet.Train(features, targets);
var predictions = elasticNet.Predict(newFeatures);

// Check which features were selected (non-zero coefficients)
var selectedFeatures = elasticNet.GetActiveFeatureIndices();

Constructors

ElasticNetRegression(ElasticNetRegressionOptions<T>?, IRegularization<T, Matrix<T>, Vector<T>>?)

Initializes a new instance of the ElasticNetRegression<T> class.

public ElasticNetRegression(ElasticNetRegressionOptions<T>? options = null, IRegularization<T, Matrix<T>, Vector<T>>? regularization = null)

Parameters

options ElasticNetRegressionOptions<T>

Configuration options for Elastic Net Regression. If null, default options are used.

regularization IRegularization<T, Matrix<T>, Vector<T>>

Optional additional regularization strategy.

Remarks

Creates a new Elastic Net Regression model with the specified options. The primary regularization is controlled by the Alpha and L1Ratio parameters in the options.

For Beginners: This creates a new Elastic Net Regression model.

You can customize the model by providing options:

// Create with default options (alpha = 1.0, l1_ratio = 0.5)
var elasticNet = new ElasticNetRegression<double>();

// Create with custom parameters (more Lasso-like)
var options = new ElasticNetRegressionOptions<double> { Alpha = 1.0, L1Ratio = 0.7 };
var elasticNet = new ElasticNetRegression<double>(options);

Properties

IterationsUsed

Gets the number of iterations used in the last training.

public int IterationsUsed { get; }

Property Value

int

NumberOfSelectedFeatures

Gets the number of non-zero coefficients (selected features).

public int NumberOfSelectedFeatures { get; }

Property Value

int

Methods

CreateNewInstance()

Creates a new instance of Elastic Net Regression with the same configuration.

protected override IFullModel<T, Matrix<T>, Vector<T>> CreateNewInstance()

Returns

IFullModel<T, Matrix<T>, Vector<T>>

A new instance with the same options.

Deserialize(byte[])

Deserializes an Elastic Net Regression model from a byte array.

public override void Deserialize(byte[] modelData)

Parameters

modelData byte[]

The byte array containing the serialized model.

GetModelMetadata()

Gets metadata about the Elastic Net Regression model.

public override ModelMetadata<T> GetModelMetadata()

Returns

ModelMetadata<T>

A ModelMetadata object containing model information.

GetModelType()

Gets the model type identifier.

protected override ModelType GetModelType()

Returns

ModelType

The ModelType enumeration value for Elastic Net Regression.

Serialize()

Serializes the Elastic Net Regression model to a byte array.

public override byte[] Serialize()

Returns

byte[]

A byte array containing the serialized model.

Train(Matrix<T>, Vector<T>)

Trains the Elastic Net Regression model using coordinate descent optimization.

public override void Train(Matrix<T> x, Vector<T> y)

Parameters

x Matrix<T>

The input features matrix where each row is a sample and each column is a feature.

y Vector<T>

The target values vector corresponding to each sample.

Remarks

Training uses coordinate descent optimization, which updates one coefficient at a time while holding others fixed. For each coefficient, the update uses a modified soft-thresholding operator that accounts for both L1 and L2 penalties.

For Beginners: This method teaches the model to make predictions from your data.

The training process uses coordinate descent similar to Lasso:

  1. Start with all coefficients at zero (or warm start from previous solution)
  2. For each feature, compute the optimal coefficient value
  3. Apply soft thresholding with L1 penalty, scaled by L2 penalty
  4. Repeat until coefficients stop changing significantly

The L2 penalty makes the solution more stable and helps when features are correlated.

Edit this page