Class SelfTeacherModel<T>

Namespace

AiDotNet.KnowledgeDistillation.Teachers

Assembly

AiDotNet.dll

Self teacher model that uses the student's own predictions from earlier training.

public class SelfTeacherModel<T> : TeacherModelBase<Vector<T>, Vector<T>, T>, ITeacherModel<Vector<T>, Vector<T>>, IJitCompilable<T>

Type Parameters

T

Inheritance

object

TeacherModelBase<Vector<T>, Vector<T>, T>

SelfTeacherModel<T>

Implements

ITeacherModel<Vector<T>, Vector<T>>

IJitCompilable<T>

Inherited Members

TeacherModelBase<Vector<T>, Vector<T>, T>.NumOps

TeacherModelBase<Vector<T>, Vector<T>, T>.OutputDimension

TeacherModelBase<Vector<T>, Vector<T>, T>.GetLogits(Vector<T>)

TeacherModelBase<Vector<T>, Vector<T>, T>.SupportsJitCompilation

TeacherModelBase<Vector<T>, Vector<T>, T>.ExportComputationGraph(List<ComputationNode<T>>)

TeacherModelBase<Vector<T>, Vector<T>, T>.CheckWrappedModelJitSupport(ITeacherModel<Vector<T>, Vector<T>>)

TeacherModelBase<Vector<T>, Vector<T>, T>.DelegateJitExport(ITeacherModel<Vector<T>, Vector<T>>, List<ComputationNode<T>>, string)

TeacherModelBase<Vector<T>, Vector<T>, T>.ThrowJitNotSupported(string, string)

TeacherModelBase<Vector<T>, Vector<T>, T>.ValidateInput(Vector<T>, string)

TeacherModelBase<Vector<T>, Vector<T>, T>.Softmax(Vector<T>, double)

TeacherModelBase<Vector<T>, Vector<T>, T>.ApplyTemperatureSoftmax(Vector<T>, double)

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

For Beginners: Self-distillation is a technique where a model learns from its own earlier predictions. This teacher can operate in two modes:

• Cached Mode: Uses pre-computed predictions from earlier epochs (no JIT support)
• Model Mode: Wraps an IJitCompilable model for dynamic predictions (JIT support available)

Constructors

SelfTeacherModel(IJitCompilable<T>, int)

Initializes a new instance of SelfTeacherModel wrapping an IJitCompilable model.

public SelfTeacherModel(IJitCompilable<T> model, int outputDimension)

Parameters

model IJitCompilable<T>

The JIT-compilable model to wrap.

outputDimension int

The output dimension of the model.

Remarks

Use this constructor when you want the teacher to generate predictions dynamically from the underlying model. JIT compilation is supported when the underlying model supports it.

You can still use CachePredictions(Vector<T>[]) to cache predictions if needed.

SelfTeacherModel(int)

Initializes a new instance of SelfTeacherModel for cached predictions mode.

public SelfTeacherModel(int outputDimension)

Parameters

outputDimension int

The output dimension of predictions.

Remarks

Use this constructor when you want to manually cache predictions via CachePredictions(Vector<T>[]) and retrieve them via GetCachedPrediction(int). JIT compilation is not supported in this mode.

Properties

OutputDimension

Gets the output dimension of the teacher model.

public override int OutputDimension { get; }

Property Value

int

SupportsJitCompilation

Gets whether this teacher supports JIT compilation.

public override bool SupportsJitCompilation { get; }

Property Value

bool

true if constructed with an IJitCompilable model that supports JIT; false if using cached predictions mode.

Methods

CachePredictions(Vector<T>[])

public void CachePredictions(Vector<T>[] predictions)

Parameters

predictions Vector<T>[]

ExportComputationGraph(List<ComputationNode<T>>)

Exports the computation graph for JIT compilation.

public override ComputationNode<T> ExportComputationGraph(List<ComputationNode<T>> inputNodes)

Parameters

inputNodes List<ComputationNode<T>>

List to populate with input nodes.

Returns

ComputationNode<T>

The output computation node.

Remarks

When constructed with an IJitCompilable model, this method delegates to the underlying model's computation graph export. When using cached predictions mode, JIT compilation is not supported because there is no computation to represent as a graph.

Exceptions

NotSupportedException

Thrown when using cached predictions mode.

GetCachedPrediction(int)

Gets a cached prediction by index.

public Vector<T> GetCachedPrediction(int index)

Parameters

index int

Index of the cached prediction.

Returns

Vector<T>

The cached logits for the specified index.

Remarks

Architecture Note: Returns raw cached logits. Temperature scaling and softmax are handled by distillation strategies, not by the teacher model.

GetLogits(Vector<T>)

Gets logits from the underlying model.

public override Vector<T> GetLogits(Vector<T> input)

Parameters

input Vector<T>

Input to the model.

Returns

Vector<T>

The logits from the underlying model.

Remarks

This method is only available when the SelfTeacherModel was constructed with an IJitCompilable model. For cached prediction mode, use GetCachedPrediction(int).

Exceptions

InvalidOperationException

Thrown when no underlying model is configured.