Class GradientBoostingClassifierOptions<T>

Namespace

AiDotNet.Models.Options

Assembly

AiDotNet.dll

Configuration options for Gradient Boosting classifier.

public class GradientBoostingClassifierOptions<T> : ClassifierOptions<T>

Type Parameters

T

The data type used for calculations.

Inheritance

object

ModelOptions

ClassifierOptions<T>

GradientBoostingClassifierOptions<T>

Inherited Members

ClassifierOptions<T>.TaskType

ClassifierOptions<T>.DecisionThreshold

ClassifierOptions<T>.UseClassWeights

ClassifierOptions<T>.ClassWeights

ModelOptions.Seed

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

Gradient Boosting builds an ensemble of trees where each tree corrects the errors of the previous ones by fitting to the gradient of the loss function.

For Beginners: Gradient Boosting is like iteratively fixing mistakes!

The process:

1. Start with a simple prediction (like the average)
2. Calculate how wrong we are (the residuals)
3. Train a tree to predict those errors
4. Add the tree's predictions (scaled down) to improve our model
5. Repeat many times

Key parameters to tune:

• n_estimators: More trees = potentially better but slower
• learning_rate: Lower values need more trees but often work better
• max_depth: Usually 3-8 works well (shallower than random forest)

Gradient Boosting often gives the best accuracy but:

• Takes longer to train
• More sensitive to hyperparameters
• More prone to overfitting if not tuned properly

Properties

LearningRate

Gets or sets the learning rate (shrinkage).

public double LearningRate { get; set; }

Property Value

double

The learning rate. Default is 0.1.

Remarks

Shrinks the contribution of each tree. Trade-off with n_estimators: Lower learning rate requires more estimators for same performance but often achieves better generalization.

Loss

Gets or sets the loss function.

public GradientBoostingLoss Loss { get; set; }

Property Value

GradientBoostingLoss

The loss function. Default is Deviance (log loss).

MaxDepth

Gets or sets the maximum depth of each tree.

public int MaxDepth { get; set; }

Property Value

int

The maximum depth. Default is 3.

Remarks

Gradient Boosting typically uses shallow trees (depth 3-8). This is different from Random Forest which often uses deeper trees.

MaxFeatures

Gets or sets the fraction of features used for each tree.

public double MaxFeatures { get; set; }

Property Value

double

The max features fraction or count. Default is 1.0 (all features).

MinImpurityDecrease

Gets or sets the minimum impurity decrease for splitting.

public double MinImpurityDecrease { get; set; }

Property Value

double

The threshold. Default is 0.0.

MinSamplesLeaf

Gets or sets the minimum samples required at a leaf.

public int MinSamplesLeaf { get; set; }

Property Value

int

The minimum samples. Default is 1.

MinSamplesSplit

Gets or sets the minimum samples required to split.

public int MinSamplesSplit { get; set; }

Property Value

int

The minimum samples. Default is 2.

NEstimators

Gets or sets the number of boosting stages.

public int NEstimators { get; set; }

Property Value

int

The number of estimators. Default is 100.

RandomState

Gets or sets the random state for reproducibility.

public int? RandomState { get; set; }

Property Value

int?

The random seed, or null for non-deterministic. Default is null.

Subsample

Gets or sets the fraction of samples used for each tree.

public double Subsample { get; set; }

Property Value

double

The subsample fraction (0.0 to 1.0). Default is 1.0.

Remarks

Values less than 1.0 result in Stochastic Gradient Boosting. This can reduce overfitting and speed up training.