Class EnsembleClassifierBase<T>

Namespace

AiDotNet.Classification.Ensemble

Assembly

AiDotNet.dll

Base class for ensemble classification methods that combine multiple classifiers.

public abstract class EnsembleClassifierBase<T> : ProbabilisticClassifierBase<T>, IProbabilisticClassifier<T>, IClassifier<T>, IFullModel<T, Matrix<T>, Vector<T>>, IModel<Matrix<T>, Vector<T>, ModelMetadata<T>>, IModelSerializer, ICheckpointableModel, IParameterizable<T, Matrix<T>, Vector<T>>, IFeatureAware, IFeatureImportance<T>, ICloneable<IFullModel<T, Matrix<T>, Vector<T>>>, IGradientComputable<T, Matrix<T>, Vector<T>>, IJitCompilable<T>

Type Parameters

T

The numeric data type used for calculations (e.g., float, double).

Inheritance

object

ClassifierBase<T>

ProbabilisticClassifierBase<T>

EnsembleClassifierBase<T>

Implements

IProbabilisticClassifier<T>

IClassifier<T>

IFullModel<T, Matrix<T>, Vector<T>>

IModel<Matrix<T>, Vector<T>, ModelMetadata<T>>

IModelSerializer

ICheckpointableModel

IParameterizable<T, Matrix<T>, Vector<T>>

IFeatureAware

IFeatureImportance<T>

ICloneable<IFullModel<T, Matrix<T>, Vector<T>>>

IGradientComputable<T, Matrix<T>, Vector<T>>

IJitCompilable<T>

Derived

AdaBoostClassifier<T>

ExtraTreesClassifier<T>

GradientBoostingClassifier<T>

RandomForestClassifier<T>

Inherited Members

ProbabilisticClassifierBase<T>.Predict(Matrix<T>)

ProbabilisticClassifierBase<T>.PredictLogProbabilities(Matrix<T>)

ProbabilisticClassifierBase<T>.ApplySoftmax(Matrix<T>)

ProbabilisticClassifierBase<T>.ApplySigmoid(Vector<T>)

ClassifierBase<T>.NumOps

ClassifierBase<T>.Engine

ClassifierBase<T>.Options

ClassifierBase<T>.Regularization

ClassifierBase<T>.NumClasses

ClassifierBase<T>.TaskType

ClassifierBase<T>.ClassLabels

ClassifierBase<T>.NumFeatures

ClassifierBase<T>.FeatureNames

ClassifierBase<T>.ExpectedParameterCount

ClassifierBase<T>.Train(Matrix<T>, Vector<T>)

ClassifierBase<T>.InferTaskType(Vector<T>)

ClassifierBase<T>.ExtractClassLabels(Vector<T>)

ClassifierBase<T>.ComputeClassWeights(Vector<T>)

ClassifierBase<T>.GetClassIndexFromLabel(T)

ClassifierBase<T>.GetModelType()

ClassifierBase<T>.Serialize()

ClassifierBase<T>.Deserialize(byte[])

ClassifierBase<T>.GetActiveFeatureIndices()

ClassifierBase<T>.IsFeatureUsed(int)

ClassifierBase<T>.SetActiveFeatureIndices(IEnumerable<int>)

ClassifierBase<T>.GetFeatureImportance()

ClassifierBase<T>.DeepCopy()

ClassifierBase<T>.CreateNewInstance()

ClassifierBase<T>.Clone()

ClassifierBase<T>.ParameterCount

ClassifierBase<T>.DefaultLossFunction

ClassifierBase<T>.SaveModel(string)

ClassifierBase<T>.LoadModel(string)

ClassifierBase<T>.SaveState(Stream)

ClassifierBase<T>.LoadState(Stream)

ClassifierBase<T>.SupportsJitCompilation

ClassifierBase<T>.ExportComputationGraph(List<ComputationNode<T>>)

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Extension Methods

DistributedExtensions.AsDistributedForHighBandwidth<T, TInput, TOutput>(IFullModel<T, TInput, TOutput>, ICommunicationBackend<T>)

DistributedExtensions.AsDistributedForLowBandwidth<T, TInput, TOutput>(IFullModel<T, TInput, TOutput>, ICommunicationBackend<T>)

DistributedExtensions.AsDistributed<T, TInput, TOutput>(IFullModel<T, TInput, TOutput>, ICommunicationBackend<T>)

DistributedExtensions.AsDistributed<T, TInput, TOutput>(IFullModel<T, TInput, TOutput>, IShardingConfiguration<T>)

Remarks

Ensemble methods combine multiple individual classifiers (base estimators) to produce a more robust and accurate prediction than any single classifier could achieve.

For Beginners: Ensemble learning is like getting opinions from a group of experts instead of just one.

Imagine you want to predict if a movie will be successful. You could:

1. Ask just one expert (single classifier)
2. Ask many experts and combine their opinions (ensemble)

The second approach is usually more reliable because:

• Individual experts may have blind spots that others don't
• Combining diverse opinions often leads to better decisions
• Errors from one expert may be corrected by others

Common ensemble strategies:

• Bagging: Train on different random subsets of data
• Boosting: Train sequentially, focusing on mistakes
• Voting: Let classifiers vote on the answer

Constructors

EnsembleClassifierBase(ClassifierOptions<T>?, IRegularization<T, Matrix<T>, Vector<T>>?, ILossFunction<T>?)

Initializes a new instance of the EnsembleClassifierBase class.

protected EnsembleClassifierBase(ClassifierOptions<T>? options = null, IRegularization<T, Matrix<T>, Vector<T>>? regularization = null, ILossFunction<T>? lossFunction = null)

Parameters

options ClassifierOptions<T>

Configuration options for the ensemble classifier.

regularization IRegularization<T, Matrix<T>, Vector<T>>

Optional regularization strategy.

lossFunction ILossFunction<T>

Optional loss function.

Properties

Estimators

The base estimators in the ensemble.

protected List<IClassifier<T>> Estimators { get; set; }

Property Value

List<IClassifier<T>>

FeatureImportances

Gets or sets the feature importances aggregated across all estimators.

public Vector<T>? FeatureImportances { get; protected set; }

Property Value

Vector<T>

NEstimators

The number of estimators in the ensemble.

public int NEstimators { get; }

Property Value

int

Methods

AggregateFeatureImportances()

Aggregates feature importances from all tree-based estimators.

protected void AggregateFeatureImportances()

ApplyGradients(Vector<T>, T)

Applies pre-computed gradients to update the model parameters.

public override void ApplyGradients(Vector<T> gradients, T learningRate)

Parameters

gradients Vector<T>

The gradient vector to apply.

learningRate T

The learning rate for the update.

Remarks

Updates parameters using: θ = θ - learningRate * gradients

For Beginners: After computing gradients (seeing which direction to move), this method actually moves the model in that direction. The learning rate controls how big of a step to take.

Distributed Training: In DDP/ZeRO-2, this applies the synchronized (averaged) gradients after communication across workers. Each worker applies the same averaged gradients to keep parameters consistent.

ComputeGradients(Matrix<T>, Vector<T>, ILossFunction<T>?)

Computes gradients of the loss function with respect to model parameters for the given data, WITHOUT updating the model parameters.

public override Vector<T> ComputeGradients(Matrix<T> input, Vector<T> target, ILossFunction<T>? lossFunction = null)

Parameters

input Matrix<T>

The input data.

target Vector<T>

The target/expected output.

lossFunction ILossFunction<T>

The loss function to use for gradient computation. If null, uses the model's default loss function.

Returns

Vector<T>

A vector containing gradients with respect to all model parameters.

Remarks

This method performs a forward pass, computes the loss, and back-propagates to compute gradients, but does NOT update the model's parameters. The parameters remain unchanged after this call.

Distributed Training: In DDP/ZeRO-2, each worker calls this to compute local gradients on its data batch. These gradients are then synchronized (averaged) across workers before applying updates. This ensures all workers compute the same parameter updates despite having different data.

For Meta-Learning: After adapting a model on a support set, you can use this method to compute gradients on the query set. These gradients become the meta-gradients for updating the meta-parameters.

For Beginners: Think of this as "dry run" training: - The model sees what direction it should move (the gradients) - But it doesn't actually move (parameters stay the same) - You get to decide what to do with this information (average with others, inspect, modify, etc.)

Exceptions

InvalidOperationException

If lossFunction is null and the model has no default loss function.

GetModelMetadata()

Gets metadata about the model.

public override ModelMetadata<T> GetModelMetadata()

Returns

ModelMetadata<T>

A ModelMetadata object containing information about the model.

Remarks

This method returns metadata about the model, including its type, feature count, complexity, description, and additional information specific to classification.

For Beginners: Model metadata provides information about the model itself, rather than the predictions it makes. This includes details about the model's structure (like how many features it uses) and characteristics (like how many classes it can predict). This information can be useful for understanding and comparing different models.

GetParameters()

Gets all model parameters as a single vector.

public override Vector<T> GetParameters()

Returns

Vector<T>

A vector containing all model parameters.

Remarks

This method returns a vector containing all model parameters for use with optimization algorithms or model comparison.

For Beginners: This method packages all the model's parameters into a single collection. This is useful for optimization algorithms that need to work with all parameters at once.

PredictProbabilities(Matrix<T>)

Aggregates predictions from all estimators in the ensemble.

public override Matrix<T> PredictProbabilities(Matrix<T> input)

Parameters

input Matrix<T>

The input features matrix.

Returns

Matrix<T>

A matrix of aggregated class probabilities.

Remarks

Default implementation averages the probability predictions from all estimators. Derived classes may override this for different aggregation strategies.

SetParameters(Vector<T>)

Sets the parameters for this model.

public override void SetParameters(Vector<T> parameters)

Parameters

parameters Vector<T>

A vector containing all model parameters.

Exceptions

ArgumentException

Thrown when the parameters vector has an incorrect length.

WithParameters(Vector<T>)

Creates a new instance of the model with specified parameters.

public override IFullModel<T, Matrix<T>, Vector<T>> WithParameters(Vector<T> parameters)

Parameters

parameters Vector<T>

A vector containing all model parameters.

Returns

IFullModel<T, Matrix<T>, Vector<T>>

A new model instance with the specified parameters.

Exceptions

ArgumentException

Thrown when the parameters vector has an incorrect length.