Table of Contents

Class CurriculumEpisodicDataLoader<T, TInput, TOutput>

Namespace
AiDotNet.Data.Loaders
Assembly
AiDotNet.dll

Provides curriculum-based episodic task sampling that progressively increases task difficulty during training.

public class CurriculumEpisodicDataLoader<T, TInput, TOutput> : EpisodicDataLoaderBase<T, TInput, TOutput>, IEpisodicDataLoader<T, TInput, TOutput>, IDataLoader<T>, IResettable, ICountable, IBatchIterable<MetaLearningTask<T, TInput, TOutput>>

Type Parameters

T

The numeric data type used for features and labels (e.g., float, double).

TInput
TOutput
Inheritance
EpisodicDataLoaderBase<T, TInput, TOutput>
CurriculumEpisodicDataLoader<T, TInput, TOutput>
Implements
IEpisodicDataLoader<T, TInput, TOutput>
IBatchIterable<MetaLearningTask<T, TInput, TOutput>>
Inherited Members
Extension Methods

Examples

var features = new Matrix<double>(1000, 784);
var labels = new Vector<double>(1000);

// Create curriculum loader: start easy (2-way 10-shot), end hard (5-way 1-shot)
var loader = new CurriculumEpisodicDataLoader<double>(
    datasetX: features,
    datasetY: labels,
    targetNWay: 5,         // Final target: 5-way
    targetKShot: 1,        // Final target: 1-shot
    queryShots: 15,        // Fixed query examples
    initialNWay: 2,        // Start easy: 2-way
    initialKShot: 10,      // Start easy: 10-shot
    seed: 42
);

int totalEpisodes = 10000;
for (int episode = 0; episode < totalEpisodes; episode++)
{
    // Update curriculum progress (0.0 to 1.0)
    double progress = (double)episode / totalEpisodes;
    loader.SetProgress(progress);

    var task = loader.GetNextTask();
    // Train on progressively harder tasks

    // At episode 0: 2-way 10-shot (easy)
    // At episode 5000: ~3-way 5-shot (medium)
    // At episode 10000: 5-way 1-shot (hard/target)
}

Remarks

The CurriculumEpisodicDataLoader implements curriculum learning for meta-learning by starting with easier tasks and progressively increasing difficulty. Easier tasks have fewer classes (lower N-way) and more examples per class (higher K-shot), while harder tasks approach the target N-way K-shot configuration. This gradual progression helps models learn more effectively by building on simpler concepts first.

For Beginners: Curriculum learning is inspired by how humans learn: - You don't start learning math with calculus - you start with counting, then addition, etc. - You don't learn a language by reading novels - you start with basic vocabulary and grammar - Complex skills are built on simpler foundations

This loader applies the same principle to meta-learning:

  • Easy tasks: 2-way 10-shot (2 classes, 10 examples each) - lots of examples, few classes
  • Medium tasks: 4-way 5-shot (4 classes, 5 examples each) - balanced difficulty
  • Hard tasks: 5-way 1-shot (5 classes, 1 example each) - few examples, many classes

How difficulty progression works:

  • Progress is tracked from 0.0 (start) to 1.0 (end)
  • At progress 0.0: Easy tasks (2-way, lots of shots)
  • At progress 0.5: Medium tasks (halfway to target)
  • At progress 1.0: Target difficulty (full N-way K-shot)

When to use this:

  • When training struggles to converge on hard tasks from the start
  • When you want to improve sample efficiency and training stability
  • When implementing human-inspired learning strategies
  • Research has shown curriculum learning often leads to better final performance

Note: You must manually update the progress as training proceeds (e.g., based on episode number, training loss, or validation accuracy).

Thread Safety: This class is not thread-safe due to internal state. Create separate instances for concurrent task generation.

Performance: Same complexity as standard EpisodicDataLoader, with minor overhead for calculating current difficulty parameters based on progress.

Constructors

CurriculumEpisodicDataLoader(Matrix<T>, Vector<T>, int, int, int, int, int, int?)

Initializes a new instance of the CurriculumEpisodicDataLoader for progressive N-way K-shot task sampling.

public CurriculumEpisodicDataLoader(Matrix<T> datasetX, Vector<T> datasetY, int targetNWay = 5, int targetKShot = 1, int queryShots = 15, int initialNWay = 2, int initialKShot = 10, int? seed = null)

Parameters

datasetX Matrix<T>

The feature matrix where each row is an example. Shape: [num_examples, num_features].

datasetY Vector<T>

The label vector containing class labels for each example. Length: num_examples.

targetNWay int

The target number of classes per task (final difficulty). Must be at least 2.

targetKShot int

The target number of support examples per class (final difficulty). Must be at least 1.

queryShots int

The number of query examples per class (constant across curriculum). Must be at least 1.

initialNWay int

The initial number of classes per task (easy start). Must be at least 2 and less than or equal to targetNWay.

initialKShot int

The initial number of support examples per class (easy start). Must be at least targetKShot.

seed int?

Optional random seed for reproducible task sampling. If null, uses a time-based seed.

Remarks

For Beginners: This constructor sets up a curriculum from easy to hard:

  • initialNWay, initialKShot: Starting difficulty (easy tasks with few classes, many examples)
  • targetNWay, targetKShot: Ending difficulty (hard tasks with many classes, few examples)
  • queryShots: Stays constant throughout (doesn't affect difficulty much)

Parameter guidelines:

  • initialNWay should be smaller than targetNWay (e.g., 2-way → 5-way)
  • initialKShot should be larger than targetKShot (e.g., 10-shot → 1-shot)
  • This creates an easy-to-hard progression

Example progressions:

  • Gentle: 3-way 5-shot → 5-way 3-shot
  • Moderate: 2-way 10-shot → 5-way 3-shot
  • Aggressive: 2-way 20-shot → 10-way 1-shot

The loader starts at progress 0.0 (easy). Call SetProgress() to advance the curriculum.

Exceptions

ArgumentNullException

Thrown when datasetX or datasetY is null.

ArgumentException

Thrown when dimensions are invalid, dataset is too small, or curriculum parameters are inconsistent.

Properties

CurriculumProgress

Gets the current curriculum progress (0.0 = easiest, 1.0 = target difficulty).

public double CurriculumProgress { get; }

Property Value

double

Methods

GetNextTaskCore()

Core implementation of curriculum-based N-way K-shot task sampling with progressive difficulty.

protected override MetaLearningTask<T, TInput, TOutput> GetNextTaskCore()

Returns

MetaLearningTask<T, TInput, TOutput>

A MetaLearningTask with difficulty based on current curriculum progress.

Remarks

This method implements progressive difficulty: 1. Calculates current N-way and K-shot based on progress (linear interpolation) 2. Randomly selects currentNWay classes 3. For each class, samples (currentKShot + queryShots) examples 4. Shuffles and splits into support and query sets 5. Constructs and returns MetaLearningTask

For Beginners: The difficulty calculation works like this:

Assume: initialNWay=2, targetNWay=5, initialKShot=10, targetKShot=1

At progress 0.0:

  • currentNWay = 2 + (5-2) * 0.0 = 2
  • currentKShot = 10 + (1-10) * 0.0 = 10
  • Result: 2-way 10-shot (easiest)

At progress 0.5:

  • currentNWay = 2 + (5-2) * 0.5 = 3.5 → 4 (rounded)
  • currentKShot = 10 + (1-10) * 0.5 = 5.5 → 5 (rounded)
  • Result: 4-way 5-shot (medium)

At progress 1.0:

  • currentNWay = 2 + (5-2) * 1.0 = 5
  • currentKShot = 10 + (1-10) * 1.0 = 1
  • Result: 5-way 1-shot (target/hardest)

SetProgress(double)

Sets the curriculum progress to control task difficulty.

public void SetProgress(double progress)

Parameters

progress double

Progress value from 0.0 (easiest) to 1.0 (target difficulty).

Remarks

For Beginners: This method controls where you are in the curriculum.

Call this method periodically during training to advance the difficulty:

  • Progress 0.0: Easiest tasks (initialNWay, initialKShot)
  • Progress 0.5: Halfway between easy and hard
  • Progress 1.0: Target tasks (targetNWay, targetKShot)

Common strategies for updating progress:

  • Linear: progress = current_episode / total_episodes
  • Step-based: increase progress every N episodes
  • Performance-based: increase when validation accuracy reaches thresholds
  • Adaptive: increase when training loss stabilizes

The curriculum automatically interpolates between initial and target difficulty.

Exceptions

ArgumentOutOfRangeException

Thrown when progress is outside [0.0, 1.0] range.

Edit this page