Table of Contents

Class MANNAlgorithm<T, TInput, TOutput>

Namespace
AiDotNet.MetaLearning.Algorithms
Assembly
AiDotNet.dll

Implementation of Memory-Augmented Neural Networks (MANN) for meta-learning.

public class MANNAlgorithm<T, TInput, TOutput> : MetaLearnerBase<T, TInput, TOutput>, IMetaLearner<T, TInput, TOutput>

Type Parameters

T

The numeric type used for calculations (e.g., float, double).

TInput

The input data type (e.g., Matrix<T>, Tensor<T>).

TOutput

The output data type (e.g., Vector<T>, Tensor<T>).

Inheritance
MetaLearnerBase<T, TInput, TOutput>
MANNAlgorithm<T, TInput, TOutput>
Implements
IMetaLearner<T, TInput, TOutput>
Inherited Members

Remarks

Memory-Augmented Neural Networks combine a neural network with an external memory matrix. The network can read from and write to this memory, enabling rapid learning by storing new information directly in memory during adaptation.

For Beginners: MANN uses an external memory like a human's working memory:

How it works:

  1. Neural network processes input and produces a key
  2. Uses key to read relevant information from external memory
  3. Combines input with memory content to make prediction
  4. During learning, writes new information to memory
  5. Memory persists across episodes for lifelong learning

Analogy: Like a student with a notebook who can:

  • Look up previous examples (reading memory)
  • Write down new important information (writing memory)
  • Use both to solve new problems

Algorithm - Memory-Augmented Neural Networks: 

# Components
controller = NeuralNetwork()        # Processes inputs and generates keys
memory = MemoryMatrix(size=N)      # External memory matrix
read_head = AttentionMechanism()   # Reads from memory
write_head = WriteMechanism()      # Writes to memory

# Episode training
for each episode:
    # Sample N-way K-shot task
    support_set = {examples_from_N_classes, K_examples_each}
    query_set = {examples_from_same_N_classes}

    # Process support set and write to memory
    for each support example (x, y):
        key = controller.encode(x)
        value = one_hot_encode(y)
        memory.write(key, value)     # Store in external memory

    # Process query set using memory
    for each query example x_q:
        # Generate key for query
        query_key = controller.encode(x_q)

        # Read from memory using attention
        read_content = read_head(query_key, memory)

        # Combine with controller output
        controller_output = controller.forward(x_q)
        combined = concatenate(controller_output, read_content)

        # Make prediction
        prediction = output_layer(combined)

    # Train end-to-end with cross-entropy loss

Key Insights:

  1. Rapid Learning: New information can be stored in memory with a single write operation, enabling one-shot learning.

  2. Continuous Learning: Memory persists across episodes, allowing the model to accumulate knowledge over time.

  3. Differentiable Memory: Both reading and writing operations are differentiable, enabling end-to-end training.

Reference: Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., & Lillicrap, T. (2016). Meta-Learning with Memory-Augmented Neural Networks. ICML.

Constructors

MANNAlgorithm(MANNOptions<T, TInput, TOutput>)

Initializes a new instance of the MANNAlgorithm class.

public MANNAlgorithm(MANNOptions<T, TInput, TOutput> options)

Parameters

options MANNOptions<T, TInput, TOutput>

The configuration options for MANN.

Remarks

For Beginners: This creates a MANN ready for meta-learning with external memory.

What MANN needs:

  • controller: Neural network that processes inputs (the MetaModel)
  • memorySize: Size of external memory matrix
  • memoryKeySize: Dimension of memory keys
  • memoryValueSize: Dimension of memory values
  • numReadHeads: How many read heads to use

What makes it different from other meta-learning:

  • Has external memory that persists across tasks
  • Can learn new patterns in one shot by writing to memory
  • Combines neural processing with explicit memory storage

Exceptions

ArgumentNullException

Thrown when options or required components are null.

ArgumentException

Thrown when configuration validation fails.

Properties

AlgorithmType

Gets the algorithm type identifier for this meta-learner.

public override MetaLearningAlgorithmType AlgorithmType { get; }

Property Value

MetaLearningAlgorithmType

Returns MANN.

Methods

Adapt(IMetaLearningTask<T, TInput, TOutput>)

Adapts to a new task by writing support examples to memory.

public override IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task)

Parameters

task IMetaLearningTask<T, TInput, TOutput>

The new task containing support set examples.

Returns

IModel<TInput, TOutput, ModelMetadata<T>>

A MANNModel that can classify using the populated memory.

Remarks

MANN adaptation is fast - it just writes support examples to memory. The returned model can then classify new examples by reading from memory.

For Beginners: When you have a new task with labeled examples, MANN stores them in its "notebook" (memory). Then it can classify new examples by looking up similar stored examples.

Exceptions

ArgumentNullException

Thrown when task is null.

MetaTrain(TaskBatch<T, TInput, TOutput>)

Performs one meta-training step using MANN's episodic training with memory.

public override T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch)

Parameters

taskBatch TaskBatch<T, TInput, TOutput>

A batch of tasks to meta-train on.

Returns

T

The average meta-loss across all tasks in the batch.

Remarks

MANN training involves writing support examples to memory and using memory reads to classify query examples:

For each task in the batch: 1. Optionally clear recent memories (based on configuration) 2. Write support set examples to external memory 3. Process query set using memory-augmented predictions 4. Compute cross-entropy loss 5. Update controller network

Exceptions

ArgumentException

Thrown when the task batch is null or empty.

Edit this page