Class DistillationForwardResult<T>

Namespace

AiDotNet.KnowledgeDistillation

Assembly

AiDotNet.dll

Encapsulates the result of a forward pass during knowledge distillation training.

public class DistillationForwardResult<T>

Type Parameters

T

The numeric type for calculations (e.g., double, float).

Inheritance

object

DistillationForwardResult<T>

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

For Beginners: When training with knowledge distillation, we need more than just the final output of a model. We also need intermediate layer activations to enable advanced distillation techniques (like feature matching or neuron selectivity). This class packages both the final output and optional intermediate activations together.

Components: - FinalOutput: The model's final predictions (e.g., class logits). Shape: [batch_size x num_classes] - IntermediateActivations: Internal layer outputs collected during forward pass (optional)

Example:

// Standard forward pass (no intermediate activations)
var result = new DistillationForwardResult<double>(finalOutput);

// Forward pass with intermediate activations collection
var activations = new IntermediateActivations<double>();
activations.Add("layer1", layer1Output);
activations.Add("layer2", layer2Output);
var result = new DistillationForwardResult<double>(finalOutput, activations);

Used By: - Teacher models: Provide reference outputs and activations - Student models: Provide outputs to compare against teacher - Distillation strategies: Compute loss from both final outputs and intermediate activations

Constructors

DistillationForwardResult(Matrix<T>)

Initializes a new instance of DistillationForwardResult with final output only.

public DistillationForwardResult(Matrix<T> finalOutput)

Parameters

finalOutput Matrix<T>

The final output matrix from the forward pass.

Remarks

Use this constructor for standard response-based distillation that only needs final outputs.

DistillationForwardResult(Matrix<T>, IntermediateActivations<T>)

Initializes a new instance of DistillationForwardResult with final output and intermediate activations.

public DistillationForwardResult(Matrix<T> finalOutput, IntermediateActivations<T> intermediateActivations)

Parameters

finalOutput Matrix<T>

The final output matrix from the forward pass.

intermediateActivations IntermediateActivations<T>

Optional intermediate layer activations.

Remarks

Use this constructor for advanced distillation strategies that need intermediate layer outputs.

Properties

FinalOutput

The final output of the model's forward pass.

public Matrix<T> FinalOutput { get; }

Property Value

Matrix<T>

Remarks

Shape: [batch_size x output_dimension]

For classification: Each row contains logits for one sample.

For regression: Each row contains predicted values for one sample.

HasIntermediateActivations

Checks if intermediate activations are available.

public bool HasIntermediateActivations { get; }

Property Value

bool

IntermediateActivations

Optional intermediate layer activations collected during the forward pass.

public IntermediateActivations<T> IntermediateActivations { get; }

Property Value

IntermediateActivations<T>

Remarks

Will be null if intermediate activations were not requested.

Only needed for advanced distillation strategies that match internal representations (e.g., feature-based distillation, attention transfer, neuron selectivity).