Class MultiModalTeacherModel<T>

Namespace

AiDotNet.KnowledgeDistillation.Teachers

Assembly

AiDotNet.dll

Multi-modal teacher that combines multiple input modalities (vision, text, audio).

public class MultiModalTeacherModel<T> : TeacherModelBase<Vector<T>, Vector<T>, T>, ITeacherModel<Vector<T>, Vector<T>>, IJitCompilable<T>

Type Parameters

T

Inheritance

object

TeacherModelBase<Vector<T>, Vector<T>, T>

MultiModalTeacherModel<T>

Implements

ITeacherModel<Vector<T>, Vector<T>>

IJitCompilable<T>

Inherited Members

TeacherModelBase<Vector<T>, Vector<T>, T>.NumOps

TeacherModelBase<Vector<T>, Vector<T>, T>.OutputDimension

TeacherModelBase<Vector<T>, Vector<T>, T>.GetLogits(Vector<T>)

TeacherModelBase<Vector<T>, Vector<T>, T>.SupportsJitCompilation

TeacherModelBase<Vector<T>, Vector<T>, T>.ExportComputationGraph(List<ComputationNode<T>>)

TeacherModelBase<Vector<T>, Vector<T>, T>.CheckWrappedModelJitSupport(ITeacherModel<Vector<T>, Vector<T>>)

TeacherModelBase<Vector<T>, Vector<T>, T>.DelegateJitExport(ITeacherModel<Vector<T>, Vector<T>>, List<ComputationNode<T>>, string)

TeacherModelBase<Vector<T>, Vector<T>, T>.ThrowJitNotSupported(string, string)

TeacherModelBase<Vector<T>, Vector<T>, T>.ValidateInput(Vector<T>, string)

TeacherModelBase<Vector<T>, Vector<T>, T>.Softmax(Vector<T>, double)

TeacherModelBase<Vector<T>, Vector<T>, T>.ApplyTemperatureSoftmax(Vector<T>, double)

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Constructors

MultiModalTeacherModel(ITeacherModel<Vector<T>, Vector<T>>[], double[]?)

public MultiModalTeacherModel(ITeacherModel<Vector<T>, Vector<T>>[] modalityTeachers, double[]? modalityWeights = null)

Parameters

modalityTeachers ITeacherModel<Vector<T>, Vector<T>>[]

modalityWeights double[]

Properties

OutputDimension

Gets the number of output dimensions (e.g., number of classes for classification).

public override int OutputDimension { get; }

Property Value

int

Remarks

For Implementers: Return the size of the output vector. For example, a 10-class classifier should return 10.

SupportsJitCompilation

Gets whether this teacher supports JIT compilation.

public override bool SupportsJitCompilation { get; }

Property Value

bool

Returns true if all modality teachers support JIT compilation; otherwise, false.

Remarks

For Beginners: Multi-modal JIT compilation is supported when all modality teachers implement IJitCompilable and support JIT. The combined computation graph weights and sums each modality's contribution.

Methods

ExportComputationGraph(List<ComputationNode<T>>)

Exports the multi-modal computation graph for JIT compilation.

public override ComputationNode<T> ExportComputationGraph(List<ComputationNode<T>> inputNodes)

Parameters

inputNodes List<ComputationNode<T>>

List to populate with input computation nodes.

Returns

ComputationNode<T>

The output computation node representing the weighted multi-modal output.

Remarks

The multi-modal graph combines each modality teacher's computation graph using weighted sum: output = w1 * modality1_output + w2 * modality2_output + ... + wN * modalityN_output

Note: All modality teachers must support JIT compilation. The combined graph enables optimized inference across all modalities in a single execution.

Exceptions

NotSupportedException

Thrown when any modality teacher does not support JIT.

GetLogits(Vector<T>)

Gets combined logits from all modality teachers.

public override Vector<T> GetLogits(Vector<T> input)

Parameters

input Vector<T>

Input data.

Returns

Vector<T>

Combined logits from all modalities.

Remarks

Architecture Note: Returns raw combined logits. Temperature scaling and softmax are handled by distillation strategies, not by the teacher model.