Class OptimizationResult<T, TInput, TOutput>

Namespace

AiDotNet.Models.Results

Assembly

AiDotNet.dll

Represents the comprehensive results of an optimization process for a symbolic model, including the best solution found, performance metrics, feature selection results, and detailed statistics for different datasets.

public class OptimizationResult<T, TInput, TOutput>

Type Parameters

T

The numeric type used for calculations, typically float or double.

TInput

TOutput

Inheritance

object

OptimizationResult<T, TInput, TOutput>

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

This class encapsulates all the information produced during the optimization of a symbolic model. It includes the best model found, its fitness score, the number of iterations performed, the history of fitness scores during optimization, the features selected for the model, detailed results for training, validation, and test datasets, fit detection analysis, and coefficient bounds. This comprehensive collection of information allows for thorough analysis of the optimization process and the resulting model.

For Beginners: This class stores everything about an optimization process and its results.

When optimizing a model:

• You start with an initial model and try to improve it
• You track how the model performs as it evolves
• You need to know which features were important
• You want to see how well it performs on different datasets

This class stores all that information, including:

• The best model found during optimization
• How good that model is (fitness score)
• How many iterations the optimization process ran
• How the model improved over time
• Which input features were selected
• Detailed performance metrics on training, validation, and test data
• Analysis of potential issues like overfitting

Having all this information in one place makes it easier to understand, evaluate, and document your optimization results.

Constructors

OptimizationResult()

Initializes a new instance of the OptimizationResult class with default values.

public OptimizationResult()

Remarks

This constructor creates a new OptimizationResult instance and initializes all properties to their default values. It obtains the appropriate numeric operations for the generic type T, creates a default symbolic model, and initializes all vectors, lists, and nested objects. This provides a clean starting point for storing optimization results. The default symbolic model is a vector model, which is a simple linear model that can be used as a safe fallback until a better model is found during optimization.

For Beginners: This constructor creates a new result object with default values.

When a new OptimizationResult is created:

• All numeric values are set to zero
• All collections (vectors, lists) are initialized as empty
• A simple default model is created as a safe fallback
• All nested result objects are initialized with their own default values

This initialization is important because:

• It ensures consistent behavior regardless of how the object is created
• It prevents potential issues with uninitialized values
• It provides a clean slate for storing optimization results

You typically won't need to call this constructor directly, as it will be used internally by the optimization process.

Properties

BestFitnessScore

Gets or sets the fitness score of the best solution.

public T BestFitnessScore { get; set; }

Property Value

T

A numeric value representing the best fitness score, initialized to zero.

Remarks

This property represents the fitness or performance score of the best solution found during optimization. The fitness score is a single numeric value that quantifies how well the model performs, typically on the training data. Higher values usually indicate better performance, though the exact interpretation depends on the specific fitness function used. Common fitness metrics include R-squared (coefficient of determination), negative mean squared error, or accuracy. The fitness score is often used as the primary criterion for comparing and selecting models during the optimization process.

For Beginners: This value tells you how well the best model performs.

The fitness score:

• Measures how well the model fits the data
• Is typically a single number that summarizes performance
• Higher values usually indicate better performance

Common fitness metrics include:

• R² (R-squared): Measures the proportion of variance explained (higher is better)
• Negative MSE (Mean Squared Error): Measures prediction error (closer to zero is better)
• Accuracy: For classification problems, the percentage of correct predictions

This value is important because:

• It tells you how good your best model is
• It can be compared with other models' fitness scores
• It's often the main criterion used during optimization

BestIntercept

Gets or sets the intercept term of the best solution.

public T BestIntercept { get; set; }

Property Value

T

A numeric value representing the intercept term, initialized to zero.

Remarks

This property represents the intercept term (also known as the bias or constant term) of the best solution. In many mathematical models, particularly linear and polynomial models, the intercept is the value of the dependent variable when all independent variables are zero. It represents the baseline value of the target variable before considering the effects of the input features. The intercept is an important parameter of the model and is often optimized along with the coefficients of the input features.

For Beginners: This is the constant term in your model equation.

The intercept:

• Is the value your model predicts when all input variables are zero
• Represents the baseline or starting point for predictions
• Is often written as the constant term in an equation

For example, in the equation y = 3x + 2, the intercept is 2.

This value is important because:

• It establishes the baseline prediction
• It can have meaningful interpretation in some domains
• It's often optimized along with other model parameters

BestSolution

Gets or sets the best model found during optimization.

public IFullModel<T, TInput, TOutput>? BestSolution { get; set; }

Property Value

IFullModel<T, TInput, TOutput>

An implementation of IFullModel<T, TInput, TOutput> representing the best solution.

Remarks

This property represents the best model found during the optimization process. The model can be either a symbolic model (such as a mathematical expression) or a more complex structure like a neural network, depending on the optimization approach used. For symbolic models, it encapsulates a mathematical expression or algorithm that can be evaluated with different inputs to produce predictions. For neural networks or other complex models, it represents the optimized network structure and parameters. The model captures the relationship discovered between the input features and the target variable during the optimization process.

For Beginners: This is the best model found during optimization.

The best solution:

• Contains the actual model structure, whether it's a mathematical formula, a neural network, or another type of model
• Represents the best model found during the optimization process
• Can be used to make predictions with new data
• May be human-readable (for symbolic models) or more complex (for neural networks)

For example:

• In symbolic regression, this might be an equation like: y = 3.2x1² + 1.7x2 - 0.5
• For a neural network, it would contain the optimized network structure and weights

This property is important because:

• It's the primary output of the optimization process
• It can be used to understand the relationships in your data (especially for symbolic models)
• It can be deployed to make predictions on new data
• It allows for flexibility in the types of models that can be optimized

CoefficientLowerBounds

Gets or sets the lower bounds for model coefficients.

public Vector<T> CoefficientLowerBounds { get; set; }

Property Value

Vector<T>

A vector of lower bound values, initialized as an empty vector.

Remarks

This property contains the lower bounds for the coefficients in the model. Coefficient bounds are constraints that limit the range of values that the model coefficients can take during optimization. Lower bounds specify the minimum allowed values for each coefficient. These bounds can be used to incorporate domain knowledge into the model, ensure physical or logical constraints are respected, or improve the stability of the optimization process. For example, in some applications, coefficients might need to be non-negative or above a certain threshold to be physically meaningful.

For Beginners: This specifies the minimum allowed values for each coefficient in your model.

The coefficient lower bounds:

• Set the minimum values that each coefficient can have
• Help constrain the model to realistic or meaningful solutions
• Can incorporate domain knowledge into the optimization process

These bounds are useful because:

• They can prevent the model from finding mathematically valid but physically impossible solutions
• They can improve optimization stability
• They allow you to enforce known constraints (like non-negativity)

For example, if you know a certain effect cannot be negative in your domain, you might set the lower bound for that coefficient to zero.

CoefficientUpperBounds

Gets or sets the upper bounds for model coefficients.

public Vector<T> CoefficientUpperBounds { get; set; }

Property Value

Vector<T>

A vector of upper bound values, initialized as an empty vector.

Remarks

This property contains the upper bounds for the coefficients in the model. Coefficient bounds are constraints that limit the range of values that the model coefficients can take during optimization. Upper bounds specify the maximum allowed values for each coefficient. These bounds can be used to incorporate domain knowledge into the model, ensure physical or logical constraints are respected, or improve the stability of the optimization process. For example, in some applications, coefficients might need to be below a certain threshold to be physically meaningful or to prevent numerical issues.

For Beginners: This specifies the maximum allowed values for each coefficient in your model.

The coefficient upper bounds:

• Set the maximum values that each coefficient can have
• Help constrain the model to realistic or meaningful solutions
• Can incorporate domain knowledge into the optimization process

These bounds are useful because:

• They can prevent the model from finding mathematically valid but physically impossible solutions
• They can improve optimization stability
• They allow you to enforce known constraints

For example, if you know a certain effect cannot exceed a specific value in your domain, you might set the upper bound for that coefficient accordingly.

FitDetectionResult

Gets or sets the results of fit detection analysis.

public FitDetectorResult<T> FitDetectionResult { get; set; }

Property Value

FitDetectorResult<T>

A FitDetectorResult<T> object containing detailed fit analysis.

Remarks

This property contains the results of fit detection analysis, which evaluates how well the model fits the data and identifies potential issues such as underfitting or overfitting. Underfitting occurs when the model is too simple to capture the underlying patterns in the data, resulting in poor performance on both training and test data. Overfitting occurs when the model is too complex and captures noise in the training data, resulting in good performance on training data but poor performance on test data. The FitDetectorResult includes the type of fit detected, a confidence level for that assessment, and recommendations for improving the model.

For Beginners: This contains an analysis of how well your model fits the data.

The fit detection result:

• Identifies if your model is underfitting, overfitting, or has a good fit
• Provides a confidence level for this assessment
• Offers specific recommendations to improve your model

This analysis helps you understand:

• If your model is too simple (underfitting)
• If your model is too complex (overfitting)
• What steps you should take to improve it

For example, if your model is overfitting, the fit detection might recommend adding regularization or reducing model complexity.

FitnessHistory

Gets or sets the history of fitness scores during optimization.

public Vector<T> FitnessHistory { get; set; }

Property Value

Vector<T>

A vector of fitness scores, initialized as an empty vector.

Remarks

This property contains the history of fitness scores recorded during the optimization process. Each element in the vector represents the fitness score of the best model at a particular iteration or checkpoint. This history can be used to analyze the convergence behavior of the optimization algorithm, identify plateaus or jumps in performance, and visualize the improvement of the model over time. It can also help in diagnosing issues such as premature convergence or oscillation.

For Beginners: This tracks how the model's performance improved over time.

The fitness history:

• Contains the fitness score at each iteration or checkpoint
• Shows how the model improved during optimization
• Can be plotted to visualize the optimization progress

This information is valuable because:

• It helps you understand how quickly the model improved
• It can show if the optimization got stuck or plateaued
• It can reveal if more iterations might have helped

For example, a steadily increasing fitness history suggests the optimization was effective, while a flat line might indicate the algorithm got stuck.

Iterations

Gets or sets the number of iterations performed during optimization.

public int Iterations { get; set; }

Property Value

int

An integer representing the number of iterations.

Remarks

This property represents the number of iterations or generations performed during the optimization process. Each iteration typically involves evaluating and potentially modifying the model to improve its performance. The number of iterations can provide insight into the computational effort expended during optimization and can be useful for comparing different optimization runs. It may also indicate whether the optimization process converged naturally or was terminated due to reaching a maximum iteration limit.

For Beginners: This tells you how many steps the optimization process took.

The iterations count:

• Represents how many times the optimization algorithm tried to improve the model
• Each iteration typically evaluates and potentially modifies the model
• Higher values usually mean more computational effort was expended

This value is useful because:

• It helps you understand how long the optimization process ran
• It can indicate whether the process converged naturally or hit a limit
• It allows you to compare the computational effort of different optimization runs

For example, an iterations value of 1000 means the optimization algorithm performed 1000 steps trying to find the best model.

SelectedFeatures

Gets or sets the list of feature vectors selected for the model.

public List<Vector<T>> SelectedFeatures { get; set; }

Property Value

List<Vector<T>>

A list of Vector<T> objects representing the selected features, initialized as an empty list.

Remarks

This property contains the list of feature vectors that were selected for use in the best model. Feature selection is the process of identifying and selecting the most relevant features (input variables) for the model, which can improve model performance, reduce overfitting, and enhance interpretability. The selected features are represented as vectors, where each vector corresponds to a feature and contains the values of that feature across all observations in the dataset. This information is useful for understanding which input variables the model considers important and for reproducing the model with new data.

For Beginners: This list shows which input variables were used in the best model.

The selected features:

• Represent the input variables that were chosen for the model
• Are stored as vectors (arrays) of values
• May be a subset of all available features if feature selection was performed

Feature selection is important because:

• Using too many features can lead to overfitting
• Some features may be irrelevant or redundant
• Models with fewer features are often more interpretable

For example, if you started with 20 potential input variables but the model only uses 5 of them, this list would contain those 5 selected features.

TestResult

Gets or sets the detailed results for the test dataset.

public OptimizationResult<T, TInput, TOutput>.DatasetResult TestResult { get; set; }

Property Value

OptimizationResult<T, TInput, TOutput>.DatasetResult

A DatasetResult object containing detailed statistics for the test dataset.

Remarks

This property contains detailed results and statistics for the test dataset, which is the portion of data set aside and not used during model training or validation. The test result includes the input features (X), target values (Y), model predictions, error statistics, prediction statistics, and basic statistics for both actual and predicted values. This information provides the most realistic estimate of how well the model will perform on completely new, unseen data and is the final evaluation metric used to assess the model's generalization ability.

For Beginners: This contains detailed information about how the model performs on completely new data.

The test result:

• Shows how well the model performs on data it has never seen before
• Provides the most realistic estimate of real-world performance
• Is the final evaluation metric for your model

This information is important because:

• It represents how your model will likely perform in production
• It's the most honest assessment of your model's capabilities
• It helps determine if your model is ready for deployment

For example, good performance on test data suggests your model will generalize well to new, unseen data in real-world applications.

TrainingResult

Gets or sets the detailed results for the training dataset.

public OptimizationResult<T, TInput, TOutput>.DatasetResult TrainingResult { get; set; }

Property Value

OptimizationResult<T, TInput, TOutput>.DatasetResult

A DatasetResult object containing detailed statistics for the training dataset.

Remarks

This property contains detailed results and statistics for the training dataset, which is the portion of data used to fit or train the model. The training result includes the input features (X), target values (Y), model predictions, error statistics, prediction statistics, and basic statistics for both actual and predicted values. This information is crucial for understanding how well the model fits the training data and for diagnosing potential issues such as underfitting or overfitting.

For Beginners: This contains detailed information about how the model performs on the training data.

The training result:

• Shows how well the model fits the data it was trained on
• Contains the actual data, predictions, and various statistics
• Helps you understand the model's basic performance

This information is important because:

• It's the baseline for evaluating your model
• It helps identify if the model is learning the patterns in the data
• It can be compared with validation and test results to detect overfitting

For example, good performance on training data but poor performance on test data would suggest the model is overfitting to the training data.

ValidationResult

Gets or sets the detailed results for the validation dataset.

public OptimizationResult<T, TInput, TOutput>.DatasetResult ValidationResult { get; set; }

Property Value

OptimizationResult<T, TInput, TOutput>.DatasetResult

A DatasetResult object containing detailed statistics for the validation dataset.

Remarks

This property contains detailed results and statistics for the validation dataset, which is the portion of data used to tune hyperparameters and evaluate the model during training. The validation result includes the input features (X), target values (Y), model predictions, error statistics, prediction statistics, and basic statistics for both actual and predicted values. This information is crucial for understanding how well the model generalizes to unseen data and for making decisions about model selection and hyperparameter tuning.

For Beginners: This contains detailed information about how the model performs on the validation data.

The validation result:

• Shows how well the model performs on data not used in training
• Helps detect overfitting (when a model performs well on training data but poorly on new data)
• Is used to guide model selection and hyperparameter tuning

This information is important because:

• It provides feedback during the model development process
• It helps you choose between different model configurations
• It gives an early indication of how well the model might perform in production

For example, if performance is significantly worse on validation data than training data, you might need to simplify your model to reduce overfitting.

Methods

DeepCopy()

Creates a deep copy of this OptimizationResult instance.

public OptimizationResult<T, TInput, TOutput> DeepCopy()

Returns

OptimizationResult<T, TInput, TOutput>

A new OptimizationResult with copied values.

WithParameters(Vector<T>)

Creates a new OptimizationResult instance with the best solution updated to use the specified parameters.

public OptimizationResult<T, TInput, TOutput> WithParameters(Vector<T> parameters)

Parameters

parameters Vector<T>

The parameters to apply to the best solution.

Returns

OptimizationResult<T, TInput, TOutput>

A new OptimizationResult with the updated model.