Table of Contents

Class ADMMOptimizer<T, TInput, TOutput>

Namespace
AiDotNet.Optimizers
Assembly
AiDotNet.dll

Implements the Alternating Direction Method of Multipliers (ADMM) optimization algorithm.

public class ADMMOptimizer<T, TInput, TOutput> : GradientBasedOptimizerBase<T, TInput, TOutput>, IGradientBasedOptimizer<T, TInput, TOutput>, IOptimizer<T, TInput, TOutput>, IModelSerializer

Type Parameters

T

The numeric type used for calculations (e.g., float, double).

TInput
TOutput
Inheritance
OptimizerBase<T, TInput, TOutput>
GradientBasedOptimizerBase<T, TInput, TOutput>
ADMMOptimizer<T, TInput, TOutput>
Implements
IGradientBasedOptimizer<T, TInput, TOutput>
IOptimizer<T, TInput, TOutput>
Inherited Members
Extension Methods

Remarks

ADMM is an algorithm for solving convex optimization problems, particularly useful for large-scale and distributed optimization. It combines the benefits of dual decomposition and augmented Lagrangian methods.

For Beginners: ADMM is like solving a complex puzzle by breaking it into smaller, manageable pieces. It's particularly good at handling problems with constraints or when you want to distribute the computation across multiple processors.

Constructors

ADMMOptimizer(IFullModel<T, TInput, TOutput>, ADMMOptimizerOptions<T, TInput, TOutput>?, IEngine?)

Initializes a new instance of the ADMMOptimizer class.

public ADMMOptimizer(IFullModel<T, TInput, TOutput> model, ADMMOptimizerOptions<T, TInput, TOutput>? options = null, IEngine? engine = null)

Parameters

model IFullModel<T, TInput, TOutput>

The model to optimize.

options ADMMOptimizerOptions<T, TInput, TOutput>

The options for configuring the ADMM optimizer.

engine IEngine

The computation engine (CPU or GPU) for vectorized operations.

Remarks

For Beginners: This sets up the ADMM optimizer with its initial configuration. You can customize various aspects of how it solves the optimization problem, or use default settings.

Methods

Deserialize(byte[])

Restores the optimizer's state from a byte array previously created by the Serialize method.

public override void Deserialize(byte[] data)

Parameters

data byte[]

The byte array containing the serialized optimizer state.

Remarks

For Beginners: This method rebuilds the optimizer's state from a saved snapshot. It's like restoring a machine to a previous configuration using a backup.

GenerateGradientCacheKey(IFullModel<T, TInput, TOutput>, TInput, TOutput)

Generates a unique key for caching gradients based on the current state of the optimizer and input data.

protected override string GenerateGradientCacheKey(IFullModel<T, TInput, TOutput> model, TInput X, TOutput y)

Parameters

model IFullModel<T, TInput, TOutput>

The symbolic model being optimized.

X TInput

The input matrix.

y TOutput

The target vector.

Returns

string

A string that uniquely identifies the current optimization state for gradient caching.

Remarks

For Beginners: This method creates a unique label for the current state of the optimization. It's used to efficiently store and retrieve calculated gradients, which helps speed up the optimization process.

GetOptions()

Retrieves the current options of the optimizer.

public override OptimizationAlgorithmOptions<T, TInput, TOutput> GetOptions()

Returns

OptimizationAlgorithmOptions<T, TInput, TOutput>

The current optimization algorithm options.

Remarks

For Beginners: This method lets you check what settings the optimizer is currently using. It's like looking at the current settings on a machine.

InitializeAdaptiveParameters()

Initializes the adaptive parameters used by the ADMM optimizer.

protected override void InitializeAdaptiveParameters()

Remarks

For Beginners: This resets the iteration count to zero, preparing the optimizer for a new optimization run.

Optimize(OptimizationInputData<T, TInput, TOutput>)

Performs the optimization process using the ADMM algorithm.

public override OptimizationResult<T, TInput, TOutput> Optimize(OptimizationInputData<T, TInput, TOutput> inputData)

Parameters

inputData OptimizationInputData<T, TInput, TOutput>

The input data for optimization, including training data and targets.

Returns

OptimizationResult<T, TInput, TOutput>

The result of the optimization process, including the best solution found.

Remarks

For Beginners: This is the main optimization process. It repeatedly updates the solution using the ADMM steps until it reaches the best possible solution or hits a stopping condition.

DataLoader Integration: This method uses the DataLoader API for epoch management. ADMM typically operates on the full dataset for the X update (which involves solving linear systems), but notifies the sampler of epoch starts using NotifyEpochStart(int) for compatibility with curriculum learning and sampling strategies.

Serialize()

Converts the current state of the optimizer into a byte array for storage or transmission.

public override byte[] Serialize()

Returns

byte[]

A byte array representing the serialized state of the optimizer.

Remarks

For Beginners: This method saves all the important information about the optimizer's current state. It's like taking a snapshot of the optimizer that can be used to recreate its exact state later.

UpdateAdaptiveParameters(OptimizationStepData<T, TInput, TOutput>, OptimizationStepData<T, TInput, TOutput>)

Updates the adaptive parameters of the optimizer based on the current and previous optimization steps.

protected override void UpdateAdaptiveParameters(OptimizationStepData<T, TInput, TOutput> currentStepData, OptimizationStepData<T, TInput, TOutput> previousStepData)

Parameters

currentStepData OptimizationStepData<T, TInput, TOutput>

Data from the current optimization step.

previousStepData OptimizationStepData<T, TInput, TOutput>

Data from the previous optimization step.

Remarks

For Beginners: This method adjusts how the optimizer behaves based on its recent performance. It can change certain parameters to help the optimizer find a better solution more quickly.

UpdateOptions(OptimizationAlgorithmOptions<T, TInput, TOutput>)

Updates the optimizer's options with new settings.

protected override void UpdateOptions(OptimizationAlgorithmOptions<T, TInput, TOutput> options)

Parameters

options OptimizationAlgorithmOptions<T, TInput, TOutput>

The new options to be applied to the optimizer.

Remarks

For Beginners: This method allows you to change the settings of the optimizer while it's running. It's like adjusting the controls on a machine that's already operating.

Exceptions

ArgumentException

Thrown when the provided options are not of the correct type.

UpdateParametersGpu(IGpuBuffer, IGpuBuffer, int, IDirectGpuBackend)

Updates parameters using GPU-accelerated ADMM.

public override void UpdateParametersGpu(IGpuBuffer parameters, IGpuBuffer gradients, int parameterCount, IDirectGpuBackend backend)

Parameters

parameters IGpuBuffer
gradients IGpuBuffer
parameterCount int
backend IDirectGpuBackend

Remarks

ADMM (Alternating Direction Method of Multipliers) requires maintaining dual variables and performing alternating minimization steps. GPU implementation is not yet available due to the multi-step iterative nature of the algorithm.

Edit this page