Table of Contents

Class iMAMLAlgorithm<T, TInput, TOutput>

Namespace
AiDotNet.MetaLearning.Algorithms
Assembly
AiDotNet.dll

Implementation of the iMAML (Implicit Model-Agnostic Meta-Learning) algorithm.

public class iMAMLAlgorithm<T, TInput, TOutput> : MetaLearnerBase<T, TInput, TOutput>, IMetaLearner<T, TInput, TOutput>

Type Parameters

T

The numeric type used for calculations (e.g., double, float).

TInput

The input data type (e.g., Matrix<T>, Tensor<T>).

TOutput

The output data type (e.g., Vector<T>, Tensor<T>).

Inheritance
MetaLearnerBase<T, TInput, TOutput>
iMAMLAlgorithm<T, TInput, TOutput>
Implements
IMetaLearner<T, TInput, TOutput>
Inherited Members

Remarks

iMAML is a memory-efficient variant of MAML that uses implicit differentiation to compute meta-gradients. Instead of backpropagating through all adaptation steps, it uses the implicit function theorem to directly compute gradients at the adapted parameters, significantly reducing memory requirements.

Key advantages over MAML: - Constant memory cost regardless of number of adaptation steps - Can use many more adaptation steps without memory issues - Often achieves better performance than first-order MAML (FOMAML)

For Beginners: iMAML solves one of MAML's biggest problems - memory usage.

The problem with MAML: - To learn from adaptation, MAML needs to remember every step - More adaptation steps = much more memory needed - This limits how much adaptation you can do

How iMAML solves it: - Uses a mathematical shortcut (implicit differentiation) - Only needs to remember the start and end points - Can do many more adaptation steps with the same memory

The implicit function theorem allows computing gradients through the adaptation process by solving: (I + lambda * H)^(-1) * g, where H is the Hessian of the inner loss and g is the gradient of the query loss. This is solved efficiently using Conjugate Gradient iteration.

Reference: Rajeswaran, A., Finn, C., Kakade, S. M., & Levine, S. (2019). Meta-learning with implicit gradients.

Constructors

iMAMLAlgorithm(iMAMLOptions<T, TInput, TOutput>)

Initializes a new instance of the iMAMLAlgorithm class.

public iMAMLAlgorithm(iMAMLOptions<T, TInput, TOutput> options)

Parameters

options iMAMLOptions<T, TInput, TOutput>

iMAML configuration options containing the model and all hyperparameters.

Examples

// Create iMAML with minimal configuration (uses all defaults)
var options = new iMAMLOptions<double, Tensor, Tensor>(myNeuralNetwork);
var imaml = new iMAMLAlgorithm<double, Tensor, Tensor>(options);

// Create iMAML with custom configuration for more adaptation steps
var options = new iMAMLOptions<double, Tensor, Tensor>(myNeuralNetwork)
{
    AdaptationSteps = 20,  // iMAML can handle many steps!
    LambdaRegularization = 2.0,
    ConjugateGradientIterations = 15
};
var imaml = new iMAMLAlgorithm<double, Tensor, Tensor>(options);

Exceptions

ArgumentNullException

Thrown when options is null.

InvalidOperationException

Thrown when required components are not set in options.

Properties

AlgorithmType

Gets the algorithm type identifier for this meta-learner.

public override MetaLearningAlgorithmType AlgorithmType { get; }

Property Value

MetaLearningAlgorithmType

Returns iMAML.

Remarks

This property identifies the algorithm as iMAML (Implicit MAML), distinguishing it from standard MAML and other meta-learning algorithms.

Methods

Adapt(IMetaLearningTask<T, TInput, TOutput>)

Adapts the meta-learned model to a new task using iMAML's inner loop optimization.

public override IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task)

Parameters

task IMetaLearningTask<T, TInput, TOutput>

The new task containing support set examples for adaptation.

Returns

IModel<TInput, TOutput, ModelMetadata<T>>

A new model instance that has been fine-tuned to the given task.

Remarks

At adaptation time, iMAML works exactly like MAML - the implicit gradient computation is only used during meta-training. This means adaptation is fast and straightforward: just run K gradient steps on the support set.

For Beginners: When you have a new task and want to adapt to it, iMAML works just like MAML. The memory savings from implicit gradients only matter during the meta-training phase, not during adaptation.

Because iMAML was meta-trained with many adaptation steps (enabled by constant memory cost), the learned initialization is often better than MAML's, leading to better adaptation performance.

Exceptions

ArgumentNullException

Thrown when task is null.

MetaTrain(TaskBatch<T, TInput, TOutput>)

Performs one meta-training step using iMAML's implicit gradient computation.

public override T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch)

Parameters

taskBatch TaskBatch<T, TInput, TOutput>

A batch of tasks to meta-train on, each containing support and query sets.

Returns

T

The average meta-loss across all tasks in the batch.

Remarks

iMAML meta-training differs from MAML in how meta-gradients are computed:

Inner Loop (Same as MAML): For each task in the batch: 1. Clone the meta-model with current meta-parameters 2. Perform K gradient descent steps on the task's support set 3. Evaluate the adapted model on the task's query set

Implicit Gradient Computation (Different from MAML): Instead of backpropagating through K steps: 1. Compute gradient of query loss w.r.t. adapted parameters 2. Solve (I + lambda * H)^(-1) * g using Conjugate Gradient 3. This gives the implicit meta-gradient with constant memory cost

For Beginners: The key difference is step 2 - instead of remembering all K adaptation steps (expensive!), iMAML uses a mathematical trick to get the same answer without storing the intermediate steps.

Exceptions

ArgumentException

Thrown when the task batch is null or empty.

InvalidOperationException

Thrown when meta-gradient computation fails.

Edit this page