Class OptimizationAlgorithmOptions<T, TInput, TOutput>

Namespace

AiDotNet.Models.Options

Assembly

AiDotNet.dll

Configuration options for optimization algorithms used in machine learning models.

public class OptimizationAlgorithmOptions<T, TInput, TOutput> : ModelOptions

Type Parameters

T

TInput

TOutput

Inheritance

object

ModelOptions

OptimizationAlgorithmOptions<T, TInput, TOutput>

Derived

AntColonyOptimizationOptions<T, TInput, TOutput>

BayesianOptimizerOptions<T, TInput, TOutput>

GeneticAlgorithmOptimizerOptions<T, TInput, TOutput>

GradientBasedOptimizerOptions<T, TInput, TOutput>

NelderMeadOptimizerOptions<T, TInput, TOutput>

ParticleSwarmOptimizationOptions<T, TInput, TOutput>

PowellOptimizerOptions<T, TInput, TOutput>

SimulatedAnnealingOptions<T, TInput, TOutput>

Inherited Members

ModelOptions.Seed

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

Optimization algorithms are methods used to find the best parameters for a machine learning model by minimizing or maximizing an objective function.

For Beginners: Think of optimization as the process of "learning" in machine learning. It's like adjusting the knobs on a radio until you get the clearest signal. These settings control how quickly and accurately the algorithm learns from your data.

Properties

BadFitPatience

Gets or sets the number of iterations to wait before adjusting parameters when the model is performing poorly.

public int BadFitPatience { get; set; }

Property Value

int

The number of iterations to wait, defaulting to 5.

Remarks

For Beginners: If your model is performing poorly, this setting determines how long to wait before making adjustments. It's like giving the model a few more chances before changing your approach.

EarlyStoppingPatience

Gets or sets the number of iterations to wait before stopping if no improvement is observed.

public int EarlyStoppingPatience { get; set; }

Property Value

int

The number of iterations to wait, defaulting to 10.

Remarks

For Beginners: This is how patient the algorithm is. If it hasn't seen any improvement for this many iterations, it will stop training. Higher values make the algorithm more patient, giving it more chances to find improvements.

ExplorationRate

Gets or sets the exploration rate for reinforcement learning and some optimization algorithms.

public double ExplorationRate { get; set; }

Property Value

double

The exploration rate, defaulting to 0.5.

Remarks

For Beginners: This balances between trying new things (exploration) and using what's already known to work (exploitation). A value of 0.5 means the algorithm spends equal time exploring new possibilities and exploiting known good solutions.

FeatureSelectionProbability

Gets or sets the probability of selecting a feature during feature selection mode.

public double FeatureSelectionProbability { get; set; }

Property Value

double

The feature selection probability, defaulting to 0.5 (50% chance). Values are automatically clamped to [0.0, 1.0] and invalid values (NaN/Infinity) are rejected.

Remarks

FeatureSelectionProbability controls how likely each feature is to be included when the optimizer is performing feature selection. Higher values mean more features will typically be selected, while lower values result in sparser feature sets. Value must be between 0 and 1.

For Beginners: When the optimizer is choosing which features (input variables) to use, this setting controls how likely each one is to be included. A value of 0.5 means each feature has a 50/50 chance of being selected. Increase this to use more features, decrease it to use fewer.

Validation: The value is automatically clamped between 0.0 and 1.0. Invalid values (NaN, Infinity) will throw an ArgumentException.

FitDetector

Gets or sets the fit detector to determine when a model has converged or is overfitting.

public IFitDetector<T, TInput, TOutput> FitDetector { get; set; }

Property Value

IFitDetector<T, TInput, TOutput>

The fit detector implementation.

Remarks

For Beginners: The fit detector helps determine when training should stop. It can detect when your model has learned as much as it can from the data, or when it's starting to memorize the training data instead of learning general patterns (overfitting).

FitnessCalculator

Gets or sets the fitness calculator to evaluate model quality during optimization.

public IFitnessCalculator<T, TInput, TOutput> FitnessCalculator { get; set; }

Property Value

IFitnessCalculator<T, TInput, TOutput>

The fitness calculator implementation.

Remarks

For Beginners: The fitness calculator assigns a score to your model based on how well it performs. This score is used by the optimization algorithm to determine which model parameters are better than others, guiding the learning process toward better solutions.

InitialLearningRate

Gets or sets the initial learning rate for the optimization algorithm.

public virtual double InitialLearningRate { get; set; }

Property Value

double

The initial learning rate, defaulting to 0.01.

Remarks

For Beginners: The learning rate controls how big of steps the algorithm takes when learning. A higher rate means bigger steps (faster learning but might overshoot), while a lower rate means smaller steps (more precise but slower learning). Think of it like adjusting the speed of learning.

InitialMomentum

Gets or sets the initial momentum value.

public double InitialMomentum { get; set; }

Property Value

double

The initial momentum, defaulting to 0.9.

Remarks

For Beginners: Momentum determines how much the previous update influences the current one. A value of 0.9 means 90% of the previous update is carried forward. Higher values make the algorithm less likely to get stuck but might make it harder to settle on the best solution.

LearningRateDecay

Gets or sets the rate at which the learning rate decreases over time.

public double LearningRateDecay { get; set; }

Property Value

double

The learning rate decay factor, defaulting to 0.99.

Remarks

For Beginners: This controls how quickly the learning rate decreases. A value of 0.99 means the learning rate becomes 99% of its previous value after each iteration, gradually slowing down the learning process.

MaxExplorationRate

Gets or sets the maximum allowed exploration rate.

public double MaxExplorationRate { get; set; }

Property Value

double

The maximum exploration rate, defaulting to 0.9.

Remarks

For Beginners: This limits how much time the algorithm spends exploring new possibilities (at most 90% with the default value). This ensures the algorithm always spends some time using what it already knows works well.

MaxIterations

Gets or sets the maximum number of iterations (epochs) for the optimization algorithm.

public int MaxIterations { get; set; }

Property Value

int

The maximum number of iterations, defaulting to 100.

Remarks

For Beginners: This is how many times the algorithm will go through your entire dataset. Each complete pass is called an "epoch". More iterations give the algorithm more chances to learn, but too many can lead to overfitting (memorizing the data instead of learning patterns).

MaxLearningRate

Gets or sets the maximum allowed learning rate.

public double MaxLearningRate { get; set; }

Property Value

double

The maximum learning rate, defaulting to 1.0.

Remarks

For Beginners: This prevents the learning rate from becoming too large, which could cause the algorithm to make wild guesses instead of learning properly.

MaxMomentum

Gets or sets the maximum allowed momentum value.

public double MaxMomentum { get; set; }

Property Value

double

The maximum momentum, defaulting to 0.99.

Remarks

For Beginners: This prevents the momentum from becoming too large, which could cause the algorithm to overshoot good solutions.

MaximumFeatures

Gets or sets the maximum number of features to consider in the model.

public int MaximumFeatures { get; set; }

Property Value

int

The maximum number of features.

Remarks

For Beginners: This limits how many characteristics (features) from your data the model can use. Setting a maximum can help prevent the model from becoming too complex.

MinExplorationRate

Gets or sets the minimum allowed exploration rate.

public double MinExplorationRate { get; set; }

Property Value

double

The minimum exploration rate, defaulting to 0.1.

Remarks

For Beginners: This ensures the algorithm always spends at least some time exploring new possibilities (at least 10% with the default value), even if it's found a good solution. This helps prevent the algorithm from getting stuck in a suboptimal solution.

MinLearningRate

Gets or sets the minimum allowed learning rate.

public double MinLearningRate { get; set; }

Property Value

double

The minimum learning rate, defaulting to 0.000001.

Remarks

For Beginners: This prevents the learning rate from becoming too small. Even if the rate keeps decreasing, it won't go below this value.

MinMomentum

Gets or sets the minimum allowed momentum value.

public double MinMomentum { get; set; }

Property Value

double

The minimum momentum, defaulting to 0.5.

Remarks

For Beginners: This prevents the momentum from becoming too small. Even if the momentum keeps decreasing, it won't go below this value.

MinimumFeatures

Gets or sets the minimum number of features to consider in the model.

public int MinimumFeatures { get; set; }

Property Value

int

The minimum number of features.

Remarks

For Beginners: Features are the characteristics or attributes in your data (like height, weight, color). This setting defines the minimum number of these characteristics the model should use.

ModelCache

Gets or sets the model cache to store and retrieve previously evaluated models.

public IModelCache<T, TInput, TOutput> ModelCache { get; set; }

Property Value

IModelCache<T, TInput, TOutput>

The model cache implementation.

Remarks

For Beginners: The model cache stores models that have already been evaluated, which can save time if the same model parameters are encountered again during optimization. This is especially useful for complex models that take a long time to evaluate.

ModelEvaluator

Gets or sets the model evaluator to use for assessing model performance.

public IModelEvaluator<T, TInput, TOutput> ModelEvaluator { get; set; }

Property Value

IModelEvaluator<T, TInput, TOutput>

The model evaluator implementation.

Remarks

For Beginners: The model evaluator is responsible for testing how well your model performs on data. It calculates metrics like accuracy or error rates that tell you how good your model is.

ModelStatsOptions

Gets or sets the options for model statistics calculation.

public ModelStatsOptions ModelStatsOptions { get; set; }

Property Value

ModelStatsOptions

The model statistics options.

Remarks

For Beginners: These settings control what statistics are calculated about your model, such as complexity metrics or feature importance. This helps you understand how your model works and what factors are most important in its predictions.

MomentumDecreaseFactor

Gets or sets the factor by which momentum decreases when performance worsens.

public double MomentumDecreaseFactor { get; set; }

Property Value

double

The momentum decrease factor, defaulting to 0.95.

Remarks

For Beginners: When the model is getting worse, momentum will be decreased by this factor. A value of 0.95 means momentum becomes 95% of its previous value, helping the model slow down when it might be heading in the wrong direction.

MomentumIncreaseFactor

Gets or sets the factor by which momentum increases when performance improves.

public double MomentumIncreaseFactor { get; set; }

Property Value

double

The momentum increase factor, defaulting to 1.05.

Remarks

For Beginners: When the model is improving, momentum will be increased by this factor. A value of 1.05 means momentum becomes 105% of its previous value, helping the model move faster in promising directions.

OptimizationMode

Gets or sets the optimization mode (feature selection, parameter tuning, or both).

public OptimizationMode OptimizationMode { get; set; }

Property Value

OptimizationMode

The optimization mode, defaulting to Both.

Remarks

OptimizationMode determines what aspects of the model the optimizer will modify. FeatureSelectionOnly only selects which features to use, ParametersOnly only adjusts model parameters, and Both allows the optimizer to do both.

For Beginners: This controls what the optimizer is allowed to change. It can choose which features (input variables) to use, adjust the model's internal settings, or do both. The default is 'Both', which gives the optimizer maximum flexibility to improve your model.

ParameterAdjustmentProbability

Gets or sets the probability of adjusting a parameter during parameter tuning mode.

public double ParameterAdjustmentProbability { get; set; }

Property Value

double

The parameter adjustment probability, defaulting to 0.3 (30% chance). Values are automatically clamped to [0.0, 1.0] and invalid values (NaN/Infinity) are rejected.

Remarks

ParameterAdjustmentProbability determines how likely each parameter is to be modified during parameter tuning. Lower values result in more conservative updates (fewer parameters changed), while higher values make more aggressive updates. Value must be between 0 and 1.

For Beginners: When the optimizer is adjusting model parameters, this controls how many of them get changed at once. A value of 0.3 means each parameter has a 30% chance of being adjusted. Lower values make smaller, more careful changes; higher values make bigger, bolder changes.

Validation: The value is automatically clamped between 0.0 and 1.0. Invalid values (NaN, Infinity) will throw an ArgumentException.

ParameterAdjustmentScale

Gets or sets the scale factor for parameter adjustments during optimization.

public double ParameterAdjustmentScale { get; set; }

Property Value

double

The parameter adjustment scale, defaulting to 0.1. Values are automatically clamped to [0.0, 1.0] and invalid values (NaN/Infinity) are rejected.

Remarks

ParameterAdjustmentScale controls how much model parameters are changed during each perturbation. Larger values result in bigger parameter changes, which can speed up exploration but may overshoot optimal values. Smaller values make smaller changes, leading to more precise but slower optimization.

For Beginners: This controls how big the changes are when the optimizer adjusts your model's parameters. A value of 0.1 means parameters change by about 10%. Increase this if optimization is too slow, decrease it if the optimizer seems to be overshooting good solutions.

Validation: The value is automatically clamped between 0.0 and 1.0. Invalid values (NaN, Infinity) will throw an ArgumentException.

PredictionOptions

Gets or sets the options for prediction statistics calculation.

public PredictionStatsOptions PredictionOptions { get; set; }

Property Value

PredictionStatsOptions

The prediction statistics options.

Remarks

For Beginners: These settings control how predictions are evaluated and what additional information (like confidence intervals) is included with predictions. This is useful for understanding how reliable your model's predictions are.

SignFlipProbability

Gets or sets the probability of flipping the sign of a parameter during perturbation.

public double SignFlipProbability { get; set; }

Property Value

double

The sign flip probability, defaulting to 0.1 (10% chance). Values are automatically clamped to [0.0, 1.0] and invalid values (NaN/Infinity) are rejected.

Remarks

SignFlipProbability determines how often parameter signs are randomly flipped during optimization. This helps the optimizer explore different regions of the solution space by allowing parameters to change direction. Value must be between 0 and 1.

For Beginners: Sometimes the optimizer tries flipping a parameter from positive to negative (or vice versa) to see if that improves the model. This setting controls how often that happens. A value of 0.1 means there's a 10% chance of flipping each time.

Validation: The value is automatically clamped between 0.0 and 1.0. Invalid values (NaN, Infinity) will throw an ArgumentException.

Tolerance

Gets or sets the convergence tolerance for the optimization algorithm.

public double Tolerance { get; set; }

Property Value

double

The convergence tolerance, defaulting to 0.000001 (1e-6).

Remarks

Tolerance defines how close the algorithm needs to get to the optimal solution before considering the optimization process complete. When the change in the objective function between iterations becomes smaller than this value, the algorithm will stop, assuming it has converged to a solution.

For Beginners: Think of tolerance as the precision level of your answer. A smaller value (like 0.000001) means the algorithm will try to find a more precise answer, but might take longer. It's like deciding whether you need to measure something to the nearest millimeter or just the nearest centimeter. If the improvement between steps becomes smaller than this value, the algorithm decides it's close enough to the best answer and stops searching.

UseAdaptiveLearningRate

Gets or sets whether to automatically adjust the learning rate during training.

public bool UseAdaptiveLearningRate { get; set; }

Property Value

bool

True to use adaptive learning rate (default), false otherwise.

Remarks

For Beginners: When enabled, the algorithm will automatically adjust how fast it learns based on its progress. It's like slowing down when you're getting close to your destination.

UseAdaptiveMomentum

Gets or sets whether to automatically adjust the momentum during training.

public bool UseAdaptiveMomentum { get; set; }

Property Value

bool

True to use adaptive momentum (default), false otherwise.

Remarks

For Beginners: Momentum helps the algorithm push through flat areas and avoid getting stuck in local minima. It's like a ball rolling downhill that can roll through small bumps. When adaptive momentum is enabled, the algorithm will adjust this effect based on its progress.

UseEarlyStopping

Gets or sets whether to use early stopping to prevent overfitting.

public bool UseEarlyStopping { get; set; }

Property Value

bool

True to use early stopping (default), false otherwise.

Remarks

For Beginners: Early stopping is like knowing when to stop studying - if your performance isn't improving anymore, it's time to stop. This prevents the model from memorizing the training data (overfitting) instead of learning general patterns.

UseExpressionTrees

Gets or sets whether to use expression trees for optimization.

public bool UseExpressionTrees { get; set; }

Property Value

bool

True to use expression trees, false otherwise (default).

Remarks

For Beginners: Expression trees are a way to represent code as data, which can make certain operations faster. This is an advanced feature that most beginners won't need to change.

Methods

CreateDefaults(OptimizerType)

public static OptimizationAlgorithmOptions<T, TInput, TOutput> CreateDefaults(OptimizerType optimizerType)

Parameters

optimizerType OptimizerType

Returns

OptimizationAlgorithmOptions<T, TInput, TOutput>