Table of Contents

Class ConjugateGradientOptimizer<T, TInput, TOutput>

Namespace
AiDotNet.Optimizers
Assembly
AiDotNet.dll

Implements the Conjugate Gradient optimization algorithm for numerical optimization problems.

public class ConjugateGradientOptimizer<T, TInput, TOutput> : GradientBasedOptimizerBase<T, TInput, TOutput>, IGradientBasedOptimizer<T, TInput, TOutput>, IOptimizer<T, TInput, TOutput>, IModelSerializer

Type Parameters

T

The numeric type used for calculations (e.g., float, double).

TInput
TOutput
Inheritance
OptimizerBase<T, TInput, TOutput>
GradientBasedOptimizerBase<T, TInput, TOutput>
ConjugateGradientOptimizer<T, TInput, TOutput>
Implements
IGradientBasedOptimizer<T, TInput, TOutput>
IOptimizer<T, TInput, TOutput>
Inherited Members
Extension Methods

Remarks

The Conjugate Gradient method is an algorithm for the numerical solution of particular systems of linear equations, namely those whose matrix is symmetric and positive-definite. It is often used to solve unconstrained optimization problems such as energy minimization.

For Beginners: This optimizer is like a smart hiker trying to find the lowest point in a hilly landscape. It uses information about the slope (gradient) and its previous steps to decide on the best direction to move next, allowing it to find the lowest point (optimal solution) more efficiently than simpler methods.

Constructors

ConjugateGradientOptimizer(IFullModel<T, TInput, TOutput>, ConjugateGradientOptimizerOptions<T, TInput, TOutput>?, IEngine?)

Initializes a new instance of the ConjugateGradientOptimizer class.

public ConjugateGradientOptimizer(IFullModel<T, TInput, TOutput> model, ConjugateGradientOptimizerOptions<T, TInput, TOutput>? options = null, IEngine? engine = null)

Parameters

model IFullModel<T, TInput, TOutput>

The model to optimize.

options ConjugateGradientOptimizerOptions<T, TInput, TOutput>

The options for configuring the Conjugate Gradient algorithm.

engine IEngine

The computation engine (CPU or GPU) for vectorized operations.

Remarks

For Beginners: This constructor sets up the Conjugate Gradient optimizer with its initial configuration. You can customize various aspects of how it works, or use default settings.

Methods

Deserialize(byte[])

Deserializes a byte array to restore the state of the Conjugate Gradient optimizer.

public override void Deserialize(byte[] data)

Parameters

data byte[]

The byte array containing the serialized state of the optimizer.

Remarks

For Beginners: This method loads a previously saved state of the optimizer. It's like restoring a saved game, allowing you to continue from where you left off or use a shared optimizer state.

Exceptions

InvalidOperationException

Thrown when deserialization of optimizer options fails.

GenerateGradientCacheKey(IFullModel<T, TInput, TOutput>, TInput, TOutput)

Generates a unique key for caching gradients in the Conjugate Gradient optimizer.

protected override string GenerateGradientCacheKey(IFullModel<T, TInput, TOutput> model, TInput X, TOutput y)

Parameters

model IFullModel<T, TInput, TOutput>

The symbolic model for which the gradient is calculated.

X TInput

The input data matrix.

y TOutput

The target vector.

Returns

string

A string representing the unique cache key.

Remarks

For Beginners: This method creates a special identifier for storing and retrieving calculated gradients. It helps avoid recalculating gradients unnecessarily, which can save a lot of computation time.

GetOptions()

Gets the current options of the Conjugate Gradient optimizer.

public override OptimizationAlgorithmOptions<T, TInput, TOutput> GetOptions()

Returns

OptimizationAlgorithmOptions<T, TInput, TOutput>

The current optimization algorithm options.

Remarks

For Beginners: This method allows you to retrieve the current settings of the Conjugate Gradient optimizer. You can use this to check or save the current configuration.

InitializeAdaptiveParameters()

Initializes the adaptive parameters used in the Conjugate Gradient algorithm.

protected override void InitializeAdaptiveParameters()

Remarks

For Beginners: This method sets up the initial state for the optimizer, including the learning rate and iteration count.

Optimize(OptimizationInputData<T, TInput, TOutput>)

Performs the main optimization process using the Conjugate Gradient algorithm.

public override OptimizationResult<T, TInput, TOutput> Optimize(OptimizationInputData<T, TInput, TOutput> inputData)

Parameters

inputData OptimizationInputData<T, TInput, TOutput>

The input data for the optimization process.

Returns

OptimizationResult<T, TInput, TOutput>

The result of the optimization process.

Remarks

For Beginners: This is the heart of the Conjugate Gradient algorithm. It iteratively improves the solution by calculating gradients, determining search directions, and updating the solution. The process continues until it reaches the maximum number of iterations or meets the stopping criteria.

DataLoader Integration: This method uses the DataLoader API for epoch management. Conjugate Gradient typically operates on the full dataset because the conjugate direction calculation (Fletcher-Reeves formula) requires consistent gradients between iterations. The method notifies the sampler of epoch starts using NotifyEpochStart(int) for compatibility with curriculum learning and sampling strategies.

Serialize()

Serializes the current state of the Conjugate Gradient optimizer into a byte array.

public override byte[] Serialize()

Returns

byte[]

A byte array representing the serialized state of the optimizer.

Remarks

For Beginners: This method saves the current state of the optimizer into a format that can be stored or transmitted. This is useful for saving progress or sharing the optimizer's state.

UpdateAdaptiveParameters(OptimizationStepData<T, TInput, TOutput>, OptimizationStepData<T, TInput, TOutput>)

Updates the adaptive parameters of the Conjugate Gradient optimizer based on the current and previous optimization steps.

protected override void UpdateAdaptiveParameters(OptimizationStepData<T, TInput, TOutput> currentStepData, OptimizationStepData<T, TInput, TOutput> previousStepData)

Parameters

currentStepData OptimizationStepData<T, TInput, TOutput>

Data from the current optimization step.

previousStepData OptimizationStepData<T, TInput, TOutput>

Data from the previous optimization step.

Remarks

For Beginners: This method adjusts the learning rate of the optimizer based on how well it's performing. If the current step improved the solution, it increases the learning rate to potentially make bigger improvements. If not, it decreases the learning rate to be more cautious.

UpdateOptions(OptimizationAlgorithmOptions<T, TInput, TOutput>)

Updates the options for the Conjugate Gradient optimizer.

protected override void UpdateOptions(OptimizationAlgorithmOptions<T, TInput, TOutput> options)

Parameters

options OptimizationAlgorithmOptions<T, TInput, TOutput>

The new options to be set.

Remarks

For Beginners: This method allows you to change the settings of the Conjugate Gradient optimizer during runtime. It checks to make sure you're providing the right kind of options specific to this algorithm.

Exceptions

ArgumentException

Thrown when the provided options are not of the correct type.

UpdateParametersGpu(IGpuBuffer, IGpuBuffer, int, IDirectGpuBackend)

Updates parameters using GPU-accelerated conjugate gradient.

public override void UpdateParametersGpu(IGpuBuffer parameters, IGpuBuffer gradients, int parameterCount, IDirectGpuBackend backend)

Parameters

parameters IGpuBuffer
gradients IGpuBuffer
parameterCount int
backend IDirectGpuBackend

Remarks

Conjugate gradient requires maintaining conjugate direction vectors. GPU implementation is not yet available due to the complexity of line search and conjugate direction updates that span multiple kernel invocations.

Edit this page