Table of Contents

Class UniformEpisodicDataLoader<T, TInput, TOutput>

Namespace
AiDotNet.Data.Loaders
Assembly
AiDotNet.dll

Provides uniform random episodic task sampling for N-way K-shot meta-learning scenarios.

public class UniformEpisodicDataLoader<T, TInput, TOutput> : EpisodicDataLoaderBase<T, TInput, TOutput>, IEpisodicDataLoader<T, TInput, TOutput>, IDataLoader<T>, IResettable, ICountable, IBatchIterable<MetaLearningTask<T, TInput, TOutput>>

Type Parameters

T

The numeric data type used for features and labels (e.g., float, double).

TInput

The input data type for tasks (e.g., Matrix<T>, Tensor<T>, double[]).

TOutput

The output data type for tasks (e.g., Vector<T>, Tensor<T>, double[]).

Inheritance
EpisodicDataLoaderBase<T, TInput, TOutput>
UniformEpisodicDataLoader<T, TInput, TOutput>
Implements
IEpisodicDataLoader<T, TInput, TOutput>
IBatchIterable<MetaLearningTask<T, TInput, TOutput>>
Inherited Members
Extension Methods

Examples

// Load a dataset (e.g., Omniglot, Mini-ImageNet)
var features = new Matrix<double>(1000, 784);  // 1000 examples, 784 features
var labels = new Vector<double>(1000);         // Class labels 0-9

// Create 5-way 3-shot loader with 10 query examples per class
var loader = new UniformEpisodicDataLoader<double>(
    datasetX: features,
    datasetY: labels,
    nWay: 5,        // 5 classes per task
    kShot: 3,       // 3 support examples per class
    queryShots: 10, // 10 query examples per class
    seed: 42        // Optional: for reproducibility
);

// Sample tasks for meta-training
for (int episode = 0; episode < 1000; episode++)
{
    var task = loader.GetNextTask();

    // Support set: [15, 784] and [15]
    Console.WriteLine($"Support: {task.SupportSetX.Shape[0]} examples");

    // Query set: [50, 784] and [50]
    Console.WriteLine($"Query: {task.QuerySetX.Shape[0]} examples");

    // Use task with MAML, Reptile, or SEAL
    model.MetaTrainStep(task);
}

Remarks

The UniformEpisodicDataLoader transforms a standard supervised learning dataset (features + labels) into a stream of meta-learning tasks using uniform random sampling. Each task contains a support set (for quick adaptation) and a query set (for evaluation), enabling algorithms like MAML, Reptile, and SEAL to learn how to learn from limited examples.

For Beginners: Meta-learning is training an AI to be a "fast learner." Instead of training a model once on lots of data, we train it on many small tasks, each with very few examples.

This loader helps create those small tasks:

  • N-way: How many different categories each task should have (e.g., 5-way = 5 classes)
  • K-shot: How many examples per category to use for learning (e.g., 3-shot = 3 examples/class)
  • Query shots: How many examples per category to use for testing (e.g., 10 queries/class)

Example: 5-way 3-shot with 10 queries

  • Support set: 5 classes × 3 examples = 15 total examples to learn from
  • Query set: 5 classes × 10 examples = 50 total examples to test on

Why this matters:

  • Mimics real-world scenarios where you have limited labeled data
  • Teaches models to generalize from very few examples
  • Enables rapid adaptation to new tasks

The same underlying dataset is resampled many times to create different tasks, each presenting a unique few-shot learning challenge.

Thread Safety: This class is not thread-safe due to internal Random state. Create separate instances for concurrent task generation.

Performance: Task creation is O(nWay × (kShot + queryShots)) for sampling and tensor construction. Preprocessing is O(n) where n is dataset size.

Constructors

UniformEpisodicDataLoader(Matrix<T>, Vector<T>, int, int, int, int?)

Initializes a new instance of the UniformEpisodicDataLoader for N-way K-shot task sampling with industry-standard defaults.

public UniformEpisodicDataLoader(Matrix<T> datasetX, Vector<T> datasetY, int nWay = 5, int kShot = 5, int queryShots = 15, int? seed = null)

Parameters

datasetX Matrix<T>

The feature matrix where each row is an example. Shape: [num_examples, num_features].

datasetY Vector<T>

The label vector containing class labels for each example. Length: num_examples.

nWay int

The number of unique classes per task. Default is 5 (standard in meta-learning).

kShot int

The number of support examples per class. Default is 5 (balanced difficulty).

queryShots int

The number of query examples per class. Default is 15 (3x kShot).

seed int?

Optional random seed for reproducible task sampling. If null, uses a time-based seed.

Remarks

For Beginners: This constructor prepares the dataset for episodic sampling.

Parameters explained:

  • datasetX: Your input features (images, embeddings, etc.) - one example per row
  • datasetY: The category/class for each example (must be integers or convertible to integers)
  • nWay: How many categories in each task (e.g., 5 means each task has 5 different classes)
  • kShot: How many training examples per category (e.g., 3 means 3 examples to learn from)
  • queryShots: How many test examples per category (e.g., 10 means 10 examples to evaluate on)
  • seed: Controls randomness - same seed = same task sequence (useful for debugging/testing)

Requirements:

  • Your dataset must have at least nWay different classes
  • Each class must have at least (kShot + queryShots) examples
  • Labels in datasetY should be integers (0, 1, 2, ...) or convertible to integers

What happens during initialization: The loader organizes your data by class, creating an index that maps each class label to the row numbers of all examples from that class. This makes sampling fast later.

Exceptions

ArgumentNullException

Thrown when datasetX or datasetY is null.

ArgumentException

Thrown when dimensions are invalid or dataset is too small.

Methods

GetNextTaskCore()

Core implementation of N-way K-shot task sampling with uniform random selection.

protected override MetaLearningTask<T, TInput, TOutput> GetNextTaskCore()

Returns

MetaLearningTask<T, TInput, TOutput>

A MetaLearningTask with randomly sampled support and query sets.

Remarks

This method performs the following steps: 1. Randomly selects N unique classes from the dataset 2. For each selected class, randomly samples (K + queryShots) examples 3. Shuffles the sampled examples to avoid ordering bias 4. Splits the shuffled examples: first K go to support set, remaining go to query set 5. Constructs tensors and returns them as a MetaLearningTask

For Beginners: Each call to this method creates a new random mini-problem (task) from your dataset.

How it works:

  1. Pick random classes: Select N different categories (e.g., for 5-way, pick classes like cat, dog, bird, fish, rabbit)
  2. Sample examples: For each class, grab K+queryShots random examples (e.g., 3+10=13 images per class)
  3. Shuffle: Mix up the examples within each class (prevents the model from learning based on example order)
  4. Split: First K examples become support (training), rest become query (testing)
  5. Package: Combine all support examples and all query examples into tensors

Important: Each task has completely different classes sampled from the dataset. The goal is to teach the model to adapt quickly to whatever classes appear in the task.

Example output shapes (5-way 3-shot, 10 queries):

  • SupportSetX: [15, num_features] - (5 classes × 3 shots)
  • SupportSetY: [15] - Labels for support set
  • QuerySetX: [50, num_features] - (5 classes × 10 queries)
  • QuerySetY: [50] - Labels for query set

Performance: O(nWay × (kShot + queryShots)) time complexity. Creates new tensor objects on each call.

Edit this page