Class IntermediateActivations<T>

Namespace

AiDotNet.KnowledgeDistillation

Assembly

AiDotNet.dll

Stores intermediate layer activations collected during a forward pass.

public class IntermediateActivations<T>

Type Parameters

T

The numeric type for calculations (e.g., double, float).

Inheritance

object

IntermediateActivations<T>

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

For Beginners: During neural network training, we often want to inspect or use the outputs from internal layers (not just the final output). These internal outputs are called "intermediate activations". This class stores them in a dictionary keyed by layer name.

Use Cases: - Feature-based distillation: Match intermediate layer outputs between teacher and student - Neuron selectivity: Analyze how individual neurons respond across a batch - Attention transfer: Transfer attention patterns from teacher to student - Debugging: Inspect what each layer is learning

Example Usage:

var activations = new IntermediateActivations<double>();
activations.Add("conv1", conv1Output);    // Store first conv layer output
activations.Add("conv2", conv2Output);    // Store second conv layer output

// Later, retrieve for analysis
var conv1Acts = activations.Get("conv1");

Properties

AllActivations

Gets all stored activations as a read-only dictionary.

public IReadOnlyDictionary<string, Matrix<T>> AllActivations { get; }

Property Value

IReadOnlyDictionary<string, Matrix<T>>

Remarks

Key = layer name, Value = activation matrix.

Note: Returns defensive copies of matrices to prevent external mutation.

LayerCount

Gets the number of layers with stored activations.

public int LayerCount { get; }

Property Value

int

Methods

Add(string, Matrix<T>)

Adds intermediate activations for a specific layer.

public void Add(string layerName, Matrix<T> activations)

Parameters

layerName string

The name or identifier of the layer (e.g., "conv1", "layer_3").

activations Matrix<T>

The activation matrix for this layer. Rows = batch samples, Columns = neurons/features.

Remarks

If the layer name already exists, it will be overwritten.

Contains(string)

Checks if activations exist for a specific layer.

public bool Contains(string layerName)

Parameters

layerName string

Returns

bool

Get(string)

Retrieves intermediate activations for a specific layer.

public Matrix<T>? Get(string layerName)

Parameters

layerName string

The name or identifier of the layer.

Returns

Matrix<T>

A defensive copy of the activation matrix, or null if the layer was not found.

Remarks

Returns a cloned matrix to prevent external mutations from corrupting stored activations.