Table of Contents

Class CNAPAlgorithm<T, TInput, TOutput>

Namespace
AiDotNet.MetaLearning.Algorithms
Assembly
AiDotNet.dll

Implementation of Conditional Neural Adaptive Processes (CNAP) for meta-learning.

public class CNAPAlgorithm<T, TInput, TOutput> : MetaLearnerBase<T, TInput, TOutput>, IMetaLearner<T, TInput, TOutput>

Type Parameters

T

The numeric type used for calculations (e.g., double, float).

TInput

The input data type (e.g., Matrix<T>, Tensor<T>).

TOutput

The output data type (e.g., Vector<T>, Tensor<T>).

Inheritance
MetaLearnerBase<T, TInput, TOutput>
CNAPAlgorithm<T, TInput, TOutput>
Implements
IMetaLearner<T, TInput, TOutput>
Inherited Members

Remarks

CNAP extends Neural Processes by conditioning on task-specific context points and learning to produce fast adaptation weights for each task. Unlike MAML, which adapts through gradient descent, CNAP learns to directly generate task-specific weight modifications from context examples.

Key Innovation: Instead of computing gradients on support sets, CNAP: 1. Encodes support set examples into a task representation 2. Uses an adaptation network to generate task-specific fast weights 3. Applies these fast weights to modify the base model for that task

For Beginners: CNAP is like having a teacher who can instantly understand a new subject from a few examples:

- MAML: Learns how to learn quickly (by finding a good starting point) - CNAP: Learns to directly generate solutions (by understanding the task)

Imagine showing someone a few examples of a new font. CNAP doesn't need to practice writing in that font (like MAML would). Instead, it understands the pattern from examples and directly modifies how it writes.

Architecture: - Encoder: Processes each context (input, output) pair into embeddings - Aggregator: Combines embeddings into a single task representation - Adaptation Network: Generates fast weights from task representation - Base Model: Modified by fast weights to perform the task

Algorithm: 

For each task batch:
  For each task:
    1. Encode context points: z_i = encoder(x_i, y_i)
    2. Aggregate to task representation: z = aggregate(z_1, ..., z_k)
    3. Generate fast weights: α = adaptation_network(z)
    4. Apply fast weights to base model: model' = apply_fast_weights(model, α)
    5. Evaluate on query set: loss = query_loss(model', query_data)
  Meta-update all networks based on query losses

Reference: Requeima, J., Gordon, J., Bronskill, J., Nowozin, S., & Turner, R. E. (2019). Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processes.

Constructors

CNAPAlgorithm(CNAPOptions<T, TInput, TOutput>)

Initializes a new instance of the CNAPAlgorithm class.

public CNAPAlgorithm(CNAPOptions<T, TInput, TOutput> options)

Parameters

options CNAPOptions<T, TInput, TOutput>

CNAP configuration options containing the model and all hyperparameters.

Examples

// Create CNAP with minimal configuration
var options = new CNAPOptions<double, Tensor, Tensor>(myNeuralNetwork);
var cnap = new CNAPAlgorithm<double, Tensor, Tensor>(options);

// Create CNAP with custom configuration
var options = new CNAPOptions<double, Tensor, Tensor>(myNeuralNetwork)
{
    RepresentationDimension = 512,
    UseAttention = true,
    NumAttentionHeads = 8,
    FastWeightMode = FastWeightApplicationMode.FiLM
};
var cnap = new CNAPAlgorithm<double, Tensor, Tensor>(options);

Exceptions

ArgumentNullException

Thrown when options is null.

InvalidOperationException

Thrown when required components are not set in options.

Properties

AlgorithmType

Gets the algorithm type identifier for this meta-learner.

public override MetaLearningAlgorithmType AlgorithmType { get; }

Property Value

MetaLearningAlgorithmType

Returns CNAP.

Remarks

This property identifies the algorithm as CNAP (Conditional Neural Adaptive Processes), a feed-forward meta-learning algorithm that generates task-specific weights from context examples without gradient-based adaptation.

Methods

Adapt(IMetaLearningTask<T, TInput, TOutput>)

Adapts the meta-learned model to a new task using CNAP's feed-forward approach.

public override IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task)

Parameters

task IMetaLearningTask<T, TInput, TOutput>

The new task containing support set examples for adaptation.

Returns

IModel<TInput, TOutput, ModelMetadata<T>>

A new model instance that has been adapted to the given task using generated fast weights.

Remarks

CNAP adaptation is extremely efficient compared to gradient-based methods:

Adaptation Process: 1. Encode all support set examples into embeddings 2. Aggregate embeddings into a task representation vector 3. Generate fast weights from the task representation 4. Apply fast weights to the base model 5. Return the adapted model (no gradient steps needed!)

For Beginners: When adapting to a new task, CNAP works like instant recognition. You show it a few examples, it immediately "understands" the task pattern, and modifies its behavior accordingly - all in a single forward pass.

Advantages over MAML at test time: - No gradient computation required (just forward passes) - Constant time regardless of number of adaptation steps - More memory efficient (no computational graph needed) - Better suited for real-time applications

Trade-offs: - May be less flexible than MAML for out-of-distribution tasks - Requires task distribution to be learnable by the adaptation network - Fast weight generation adds parameters that need to be trained

Exceptions

ArgumentNullException

Thrown when task is null.

MetaTrain(TaskBatch<T, TInput, TOutput>)

Performs one meta-training step using CNAP's feed-forward adaptation approach.

public override T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch)

Parameters

taskBatch TaskBatch<T, TInput, TOutput>

A batch of tasks to meta-train on, each containing support and query sets.

Returns

T

The average meta-loss across all tasks in the batch (evaluated on query sets).

Remarks

CNAP meta-training differs fundamentally from gradient-based methods like MAML:

CNAP Approach (Feed-Forward): 1. Encode context points from support set into embeddings 2. Aggregate embeddings into a single task representation 3. Generate fast weights directly from the task representation 4. Apply fast weights to modify base model 5. Evaluate on query set and compute loss 6. Update encoder and adaptation networks based on query loss

Key Differences from MAML: - No gradient steps during inner loop (feed-forward only) - Fast weights are generated, not computed through gradients - More efficient at test time (no gradient computation needed) - Can be more sample-efficient for similar task distributions

For Beginners: While MAML learns "how to learn quickly," CNAP learns "how to directly understand tasks." CNAP looks at the support set examples and generates a task-specific modification in one shot, without iterating. This is faster at test time but requires the task distribution to be well-structured enough for the adaptation network to learn patterns.

Exceptions

ArgumentException

Thrown when the task batch is null or empty.

InvalidOperationException

Thrown when meta-gradient computation fails.

Edit this page