Interface ICurriculumDistillationStrategy<T>

Namespace

AiDotNet.KnowledgeDistillation.Strategies

Assembly

AiDotNet.dll

Interface for curriculum distillation strategies that progressively adjust training difficulty.

public interface ICurriculumDistillationStrategy<T>

Type Parameters

T

The numeric type for calculations (e.g., double, float).

Remarks

For Beginners: Curriculum learning is inspired by how humans learn - starting with easy concepts and gradually increasing difficulty. This interface defines strategies that control this progression during knowledge distillation.

Key Concepts: - **Progressive Difficulty**: Training difficulty increases (or decreases) over time - **Sample Filtering**: Only include samples appropriate for current curriculum stage - **Temperature Progression**: Temperature adjusts based on training progress

Common Curriculum Strategies: - **Easy-to-Hard**: Start with simple samples, gradually add harder ones - **Hard-to-Easy**: Start with challenging samples, then easier ones (for fine-tuning) - **Paced Learning**: Combine difficulty-based and time-based progression

When to Use: - Training data has clear difficulty levels - Student model benefits from structured learning progression - You want to prevent overwhelming the student early in training

Properties

CurriculumProgress

Gets the current curriculum progress as a ratio (0.0 to 1.0).

double CurriculumProgress { get; }

Property Value

double

Remarks

0.0 = Beginning of curriculum, 1.0 = End of curriculum. Progress determines which samples are included and what temperature is used.

MaxTemperature

Gets the maximum temperature for the curriculum range.

double MaxTemperature { get; }

Property Value

double

Remarks

For Easy-to-Hard: This is the starting temperature (easier samples). For Hard-to-Easy: This is the ending temperature (easier samples).

MinTemperature

Gets the minimum temperature for the curriculum range.

double MinTemperature { get; }

Property Value

double

Remarks

For Easy-to-Hard: This is the ending temperature (harder samples). For Hard-to-Easy: This is the starting temperature (harder samples).

TotalSteps

Gets the total number of steps/epochs in the curriculum.

int TotalSteps { get; }

Property Value

int

Remarks

Defines the duration of the curriculum progression. After this many steps, all samples (regardless of difficulty) are included.

Methods

ComputeCurriculumTemperature()

Computes the curriculum-adjusted temperature based on current progress.

double ComputeCurriculumTemperature()

Returns

double

Temperature value for current curriculum stage.

Remarks

For Implementers: This method should return a temperature that progresses from MaxTemperature to MinTemperature (Easy-to-Hard) or vice versa (Hard-to-Easy) based on CurriculumProgress.

GetSampleDifficulty(int)

Gets the difficulty score for a specific sample, if set.

double? GetSampleDifficulty(int sampleIndex)

Parameters

sampleIndex int

Index of the sample.

Returns

double?

Difficulty score, or null if not set.

SetSampleDifficulty(int, double)

Sets the difficulty score for a specific training sample.

void SetSampleDifficulty(int sampleIndex, double difficulty)

Parameters

sampleIndex int

Index of the sample.

difficulty double

Difficulty score (0.0 = easy, 1.0 = hard).

Remarks

For Beginners: Assign difficulty scores before training to control which samples appear at each curriculum stage.

Difficulty can be based on: - Model uncertainty on validation set - Label complexity (e.g., number of classes) - Expert annotation - Automatic difficulty estimation

ShouldIncludeSample(int)

Determines if a sample should be included in the current curriculum stage.

bool ShouldIncludeSample(int sampleIndex)

Parameters

sampleIndex int

Index of the sample.

Returns

bool

True if the sample should be included in current training.

Remarks

For Beginners: Use this to filter your training batch based on the current curriculum stage.

Example:

foreach (var (sample, index) in trainingSamples.WithIndex())
{
    if (!curriculumStrategy.ShouldIncludeSample(index))
        continue; // Skip this sample for now
// ... train on this sample

}

UpdateProgress(int)

Updates the current curriculum progress.

void UpdateProgress(int step)

Parameters

step int

Current training step/epoch (0 to TotalSteps-1).

Remarks

For Beginners: Call this at the beginning of each epoch or training iteration to advance the curriculum. The strategy will adjust temperature and sample filtering accordingly.

Example:

for (int epoch = 0; epoch < 100; epoch++)
{
    curriculumStrategy.UpdateProgress(epoch);
    // ... training loop
}