Table of Contents

Class ANILAlgorithm<T, TInput, TOutput>

Namespace
AiDotNet.MetaLearning.Algorithms
Assembly
AiDotNet.dll

Implementation of Almost No Inner Loop (ANIL) meta-learning algorithm.

public class ANILAlgorithm<T, TInput, TOutput> : MetaLearnerBase<T, TInput, TOutput>, IMetaLearner<T, TInput, TOutput>

Type Parameters

T

The numeric type used for calculations (e.g., double, float).

TInput

The input data type (e.g., Matrix<T>, Tensor<T>).

TOutput

The output data type (e.g., Vector<T>, Tensor<T>).

Inheritance
MetaLearnerBase<T, TInput, TOutput>
ANILAlgorithm<T, TInput, TOutput>
Implements
IMetaLearner<T, TInput, TOutput>
Inherited Members

Remarks

ANIL is a simplified version of MAML that only adapts the classification head during inner-loop adaptation while keeping the feature extractor (body) frozen. This significantly reduces computation while often maintaining competitive performance.

Key Insight: Most of the "learning to learn" ability in MAML comes from learning a good feature representation, not from adapting the entire network. By only adapting the final classification layer, ANIL achieves:

  • Much faster adaptation (fewer parameters to update)
  • Lower memory usage (no gradients stored for the body)
  • Comparable performance to full MAML in many scenarios

For Beginners: Think of a neural network as having two parts:

1. Body (Feature Extractor): Like learning to see and understand images 2. Head (Classifier): Like learning which button to press for each category

ANIL says: "The 'seeing' part is general enough - we just need to learn which button to press for each new task!" So it only updates the button-pressing part (head) and keeps the seeing part (body) fixed during adaptation.

Algorithm (MAML-style with head-only adaptation): 

For each task batch:
  For each task:
    1. Clone the classification head parameters
    2. Freeze body, only compute gradients for head
    3. For each adaptation step:
       a. Forward pass through body (frozen) + head
       b. Compute loss on support set
       c. Update ONLY head parameters
    4. Evaluate adapted head on query set
  Meta-update: body (outer loop) + head initialization (inner loop starting point)

Reference: Raghu, A., Raghu, M., Bengio, S., & Vinyals, O. (2020). Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML.

Constructors

ANILAlgorithm(ANILOptions<T, TInput, TOutput>)

Initializes a new instance of the ANILAlgorithm class.

public ANILAlgorithm(ANILOptions<T, TInput, TOutput> options)

Parameters

options ANILOptions<T, TInput, TOutput>

ANIL configuration options containing the model and all hyperparameters.

Examples

// Create ANIL with minimal configuration
var options = new ANILOptions<double, Tensor, Tensor>(myNeuralNetwork);
var anil = new ANILAlgorithm<double, Tensor, Tensor>(options);

// Create ANIL with custom configuration
var options = new ANILOptions<double, Tensor, Tensor>(myNeuralNetwork)
{
    AdaptationSteps = 5,
    InnerLearningRate = 0.01,
    NumClasses = 5,
    FeatureDimension = 512
};
var anil = new ANILAlgorithm<double, Tensor, Tensor>(options);

Exceptions

ArgumentNullException

Thrown when options is null.

InvalidOperationException

Thrown when required components are not set in options.

Properties

AlgorithmType

Gets the algorithm type identifier for this meta-learner.

public override MetaLearningAlgorithmType AlgorithmType { get; }

Property Value

MetaLearningAlgorithmType

Returns ANIL.

Remarks

This property identifies the algorithm as ANIL (Almost No Inner Loop), a simplified MAML variant that only adapts the classification head.

Methods

Adapt(IMetaLearningTask<T, TInput, TOutput>)

Adapts the meta-learned model to a new task by only updating the classification head.

public override IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task)

Parameters

task IMetaLearningTask<T, TInput, TOutput>

The new task containing support set examples for adaptation.

Returns

IModel<TInput, TOutput, ModelMetadata<T>>

A new model instance that has been adapted to the given task with an updated head.

Remarks

ANIL adaptation is significantly faster than full MAML adaptation because only the classification head parameters are updated.

Adaptation Process: 1. Clone the meta-learned head parameters 2. Optionally reinitialize head (if configured) 3. For each adaptation step: a. Forward pass: frozen body + current head b. Compute loss on support set c. Update head parameters with gradient descent 4. Return model with frozen body + adapted head

For Beginners: When you give ANIL a new task: 1. It keeps its "vision" (feature extractor) exactly the same 2. It only retrains the "decision maker" (classifier head) 3. This is like teaching someone who already knows how to see, just teaching them what labels to assign to things

Speed Advantage: If your network has 1 million parameters and the head has only 5000, ANIL updates 200x fewer parameters per adaptation step!

Exceptions

ArgumentNullException

Thrown when task is null.

MetaTrain(TaskBatch<T, TInput, TOutput>)

Performs one meta-training step using ANIL's head-only adaptation approach.

public override T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch)

Parameters

taskBatch TaskBatch<T, TInput, TOutput>

A batch of tasks to meta-train on, each containing support and query sets.

Returns

T

The average meta-loss across all tasks in the batch (evaluated on query sets).

Remarks

ANIL meta-training is a simplified version of MAML:

ANIL Inner Loop (per task): 1. Clone the head parameters from the meta-learned initialization 2. Keep body parameters frozen (no gradients computed for body) 3. Perform gradient descent on head parameters using support set 4. Evaluate adapted model on query set

ANIL Outer Loop: 1. Accumulate gradients from all tasks' query losses 2. Update body parameters to improve feature extraction 3. Update head initialization to provide better starting point

Key Differences from MAML: - Only head parameters are updated in inner loop - Body parameters are only updated in outer loop - First-order approximation is typically used - Much faster per-iteration (fewer parameters to track)

For Beginners: ANIL learns two things: 1. A good feature extractor that works for all tasks (updated in outer loop) 2. A good starting point for the classifier head (adapted per task)

During adaptation, only the classifier changes - the "vision system" stays fixed. This is much faster because the classifier is much smaller than the full network.

Exceptions

ArgumentException

Thrown when the task batch is null or empty.

InvalidOperationException

Thrown when meta-gradient computation fails.

Edit this page