Interface IDistillationStrategy<T, TInput, TOutput>

Namespace

AiDotNet.ContinualLearning.Interfaces

Assembly

AiDotNet.dll

Extended strategy interface for knowledge distillation-based strategies.

public interface IDistillationStrategy<T, TInput, TOutput> : IContinualLearningStrategy<T, TInput, TOutput>

Type Parameters

T

The numeric type used for calculations.

TInput

The input data type.

TOutput

The output data type.

Inherited Members

IContinualLearningStrategy<T, TInput, TOutput>.Name

IContinualLearningStrategy<T, TInput, TOutput>.RequiresMemoryBuffer

IContinualLearningStrategy<T, TInput, TOutput>.ModifiesArchitecture

IContinualLearningStrategy<T, TInput, TOutput>.MemoryUsageBytes

IContinualLearningStrategy<T, TInput, TOutput>.PrepareForTask(IFullModel<T, TInput, TOutput>, IDataset<T, TInput, TOutput>)

IContinualLearningStrategy<T, TInput, TOutput>.ComputeRegularizationLoss(IFullModel<T, TInput, TOutput>)

IContinualLearningStrategy<T, TInput, TOutput>.AdjustGradients(Vector<T>)

IContinualLearningStrategy<T, TInput, TOutput>.FinalizeTask(IFullModel<T, TInput, TOutput>)

IContinualLearningStrategy<T, TInput, TOutput>.Reset()

IContinualLearningStrategy<T, TInput, TOutput>.Save(string)

IContinualLearningStrategy<T, TInput, TOutput>.Load(string)

IContinualLearningStrategy<T, TInput, TOutput>.GetMetrics()

IContinualLearningStrategy<T, TInput, TOutput>.IsCompatibleWith(IFullModel<T, TInput, TOutput>)

IContinualLearningStrategy<T, TInput, TOutput>.GetIncompatibilityReason(IFullModel<T, TInput, TOutput>)

Properties

DistillationWeight

Gets the weight for distillation loss vs task loss.

T DistillationWeight { get; }

Property Value

T

TeacherModel

Gets the teacher model (frozen copy from before current task).

IFullModel<T, TInput, TOutput>? TeacherModel { get; }

Property Value

IFullModel<T, TInput, TOutput>

Temperature

Gets the distillation temperature.

T Temperature { get; }

Property Value

T

Methods

ComputeDistillationLoss(TOutput, TOutput)

Computes the distillation loss between teacher and student outputs.

T ComputeDistillationLoss(TOutput teacherOutput, TOutput studentOutput)

Parameters

teacherOutput TOutput

Output from the teacher model.

studentOutput TOutput

Output from the current student model.

Returns

T

The distillation loss.