Table of Contents

Class IsotonicRegression<T>

Namespace
AiDotNet.Regression
Assembly
AiDotNet.dll

Implements an Isotonic Regression model, which fits a free-form line to data with the constraint that the fitted line must be non-decreasing (monotonically increasing).

public class IsotonicRegression<T> : NonLinearRegressionBase<T>, INonLinearRegression<T>, IRegression<T>, IFullModel<T, Matrix<T>, Vector<T>>, IModel<Matrix<T>, Vector<T>, ModelMetadata<T>>, IModelSerializer, ICheckpointableModel, IParameterizable<T, Matrix<T>, Vector<T>>, IFeatureAware, IFeatureImportance<T>, ICloneable<IFullModel<T, Matrix<T>, Vector<T>>>, IGradientComputable<T, Matrix<T>, Vector<T>>, IJitCompilable<T>

Type Parameters

T

The numeric type used for calculations, typically float or double.

Inheritance
IsotonicRegression<T>
Implements
IFullModel<T, Matrix<T>, Vector<T>>
IModel<Matrix<T>, Vector<T>, ModelMetadata<T>>
IParameterizable<T, Matrix<T>, Vector<T>>
ICloneable<IFullModel<T, Matrix<T>, Vector<T>>>
IGradientComputable<T, Matrix<T>, Vector<T>>
Inherited Members
Extension Methods

Remarks

Isotonic Regression is a form of nonlinear regression that fits a non-decreasing function to data. Unlike many regression techniques, it makes minimal assumptions about the shape of the function besides monotonicity (that the function doesn't decrease as the input increases). This makes it particularly useful for calibrating probability estimates from other models or for situations where a monotonic relationship is expected between the input and output variables.

For Beginners: Isotonic Regression creates a "stair-step" function that only goes up, never down.

Imagine you're drawing a line through points on a graph, but with two key rules:

  • The line can go up or stay flat, but it can never go down
  • The line should stick as close as possible to all the data points

This model is useful when you know that as one value increases, the other should never decrease. For example:

  • As study time increases, test scores shouldn't decrease
  • As price increases, demand shouldn't increase
  • As age increases, height (for children) shouldn't decrease

Unlike a straight line (linear regression), Isotonic Regression can capture more complex relationships while still maintaining this "never decreasing" property.

Constructors

IsotonicRegression(NonLinearRegressionOptions?, IRegularization<T, Matrix<T>, Vector<T>>?)

Initializes a new instance of the IsotonicRegression<T> class.

public IsotonicRegression(NonLinearRegressionOptions? options = null, IRegularization<T, Matrix<T>, Vector<T>>? regularization = null)

Parameters

options NonLinearRegressionOptions

Optional configuration options for the nonlinear regression algorithm.

regularization IRegularization<T, Matrix<T>, Vector<T>>

Optional regularization strategy to prevent overfitting.

Remarks

This constructor creates a new Isotonic Regression model with the specified options and regularization strategy. If no options are provided, default values are used. If no regularization is specified, no regularization is applied.

For Beginners: This is how you create a new Isotonic Regression model.

The constructor is quite simple because Isotonic Regression doesn't have many configuration options compared to more complex models. You can specify:

  • Options: General settings for nonlinear regression
  • Regularization: Helps prevent the model from being too sensitive to small variations in the data

If you don't specify these parameters, the model will use reasonable default settings.

Example:

// Create an Isotonic Regression model with default settings
var isoReg = new IsotonicRegression<double>();

Methods

CreateInstance()

Creates a new instance of the IsotonicRegression with the same configuration as the current instance.

protected override IFullModel<T, Matrix<T>, Vector<T>> CreateInstance()

Returns

IFullModel<T, Matrix<T>, Vector<T>>

A new IsotonicRegression instance with the same options and regularization as the current instance.

Remarks

This method creates a new instance of the IsotonicRegression model with the same configuration options and regularization settings as the current instance. This is useful for model cloning, ensemble methods, or cross-validation scenarios where multiple instances of the same model with identical configurations are needed.

For Beginners: This method creates a fresh copy of the model's blueprint.

When you need multiple versions of the same type of model with identical settings:

  • This method creates a new, empty model with the same configuration
  • It's like making a copy of a recipe before you start cooking
  • The new model has the same settings but no trained data
  • This is useful for techniques that need multiple models, like cross-validation

For example, when testing your model on different subsets of data, you'd want each test to use a model with identical settings.

Deserialize(byte[])

Loads a previously serialized Isotonic Regression model from a byte array.

public override void Deserialize(byte[] modelData)

Parameters

modelData byte[]

The byte array containing the serialized model.

Remarks

This method reconstructs an Isotonic Regression model from a byte array that was previously created using the Serialize method. It restores the base class data, input values, and target values, allowing the model to be used for predictions without retraining.

For Beginners: This method loads a previously saved model from a sequence of bytes.

Deserialization allows you to:

  • Load a model that was saved earlier
  • Use a model without having to retrain it
  • Share models between different applications

When you deserialize a model:

  • The input and output values from training are recovered
  • The model is ready to make predictions immediately
  • You don't need to go through the training process again

Example:

// Load from a file
byte[] modelData = File.ReadAllBytes("isotonicRegression.model");

// Deserialize the model
var isoReg = new IsotonicRegression<double>();
isoReg.Deserialize(modelData);

// Now you can use the model for predictions
var predictions = isoReg.Predict(newFeatures);

GetModelType()

Gets the model type of the Isotonic Regression model.

protected override ModelType GetModelType()

Returns

ModelType

The model type enumeration value.

OptimizeModel(Matrix<T>, Vector<T>)

Optimizes the Isotonic Regression model using the Pool Adjacent Violators (PAV) algorithm.

protected override void OptimizeModel(Matrix<T> x, Vector<T> y)

Parameters

x Matrix<T>

The feature matrix of training samples.

y Vector<T>

The target vector of training samples.

Remarks

This method implements the Pool Adjacent Violators (PAV) algorithm, which is the standard optimization technique for Isotonic Regression. The algorithm starts by assigning each target value to its corresponding input value, then repeatedly merges adjacent violators (points that violate the monotonicity constraint) by replacing them with their weighted average. This process continues until the fitted values are monotonically increasing.

For Beginners: This method does the actual work of finding the best "stair-step" function.

The algorithm works like this:

  1. Start by assuming each input point maps directly to its output value
  2. Look for any places where the function would go down (violating the "never decreasing" rule)
  3. When it finds such places, it "pools" those points together by replacing them with their average value
  4. It keeps doing this until there are no more violations and the function only goes up or stays flat

This approach is called the "Pool Adjacent Violators" algorithm, and it guarantees finding the monotonically increasing function that best fits your data.

Predict(Matrix<T>)

Predicts target values for the provided input features using the trained Isotonic Regression model.

public override Vector<T> Predict(Matrix<T> input)

Parameters

input Matrix<T>

A matrix where each row represents a sample to predict and each column represents a feature. For Isotonic Regression, typically only the first column is used.

Returns

Vector<T>

A vector of predicted values corresponding to each input sample.

Remarks

This method predicts target values for new input data by finding the nearest support vector for each input sample and retrieving the corresponding fitted value. Since Isotonic Regression typically works with 1D input, only the first column of the input matrix is used for predictions.

For Beginners: This method uses your trained model to make predictions on new data.

To make a prediction for a new input value, the model:

  1. Looks at the input value (typically just one number)
  2. Finds the closest value from the training data
  3. Returns the corresponding output value from the fitted "stair-step" function

This approach ensures that predictions follow the same monotonic (never decreasing) pattern that was learned during training.

Example:

// Make predictions
var predictions = isoReg.Predict(newFeatures);

PredictSingle(Vector<T>)

Predicts the target value for a single input feature vector.

protected override T PredictSingle(Vector<T> input)

Parameters

input Vector<T>

The feature vector of the sample to predict.

Returns

T

The predicted value for the input sample.

Remarks

This method predicts the target value for a single input feature vector by extracting the first element (since Isotonic Regression typically works with 1D input), finding the nearest support vector, and retrieving the corresponding fitted value.

For Beginners: This is the method that makes a prediction for a single input value.

For a given input:

  1. It takes just the first feature (Isotonic Regression usually works with one input variable)
  2. It finds the closest matching point from the training data
  3. It returns the corresponding output value from the fitted function

This ensures that the prediction follows the monotonic relationship learned during training.

Serialize()

Serializes the Isotonic Regression model to a byte array for storage or transmission.

public override byte[] Serialize()

Returns

byte[]

A byte array containing the serialized model.

Remarks

This method converts the Isotonic Regression model into a byte array that can be stored in a file, database, or transmitted over a network. The serialized data includes the base class data, input values, and target values used during training.

For Beginners: This method saves your trained model as a sequence of bytes.

Serialization allows you to:

  • Save your model to a file
  • Store your model in a database
  • Send your model over a network
  • Keep your model for later use without having to retrain it

The serialized data includes:

  • The input values from your training data
  • The corresponding output values
  • All the information needed to recreate the model exactly as it was

Example:

// Serialize the model
byte[] modelData = isoReg.Serialize();

// Save to a file
File.WriteAllBytes("isotonicRegression.model", modelData);

Train(Matrix<T>, Vector<T>)

Trains the Isotonic Regression model using the provided input features and target values.

public override void Train(Matrix<T> x, Vector<T> y)

Parameters

x Matrix<T>

A matrix where each row represents a sample and each column represents a feature. For Isotonic Regression, typically only the first column is used.

y Vector<T>

A vector of target values corresponding to each sample in x.

Remarks

This method trains the Isotonic Regression model by validating inputs, extracting the first column of the feature matrix (since Isotonic Regression typically works with 1D input), and optimizing the model using the Pool Adjacent Violators (PAV) algorithm. The model learns a non-decreasing function that best fits the relationship between the input feature and the target values.

For Beginners: This method teaches the model how to make predictions using your data.

During training, the model:

  1. Takes your input data (typically just one feature/column)
  2. Links each input value with its corresponding output value
  3. Adjusts the relationship to ensure it only goes up or stays flat, never down
  4. Uses a special algorithm (called Pool Adjacent Violators) to find the best-fitting monotonic function

After training, the model will be ready to make predictions on new data.

Example:

// Train the model
isoReg.Train(features, targets);
Edit this page