Interface IProjectorHead<T>

Namespace

AiDotNet.SelfSupervisedLearning

Assembly

AiDotNet.dll

Defines the contract for projection heads used in self-supervised learning.

public interface IProjectorHead<T>

Type Parameters

T

The numeric type used for computations (typically float or double).

Remarks

For Beginners: A projection head is a small neural network that transforms encoder outputs into a space optimized for the SSL loss. After pretraining, the projection head is typically discarded and only the encoder is used for downstream tasks.

Why use a projection head?

• Prevents the contrastive loss from degrading encoder representations
• Allows the encoder to keep more general information
• Empirically shown to significantly improve downstream performance

Common architectures:

• Linear: Single linear layer (simplest)
• MLP: 2-3 layer MLP with BatchNorm and ReLU (most common)
• Symmetric: Predictor network in BYOL/SimSiam

Properties

HiddenDimension

Gets the hidden dimension (for MLP projectors).

int? HiddenDimension { get; }

Property Value

int?

Remarks

Typical values: 2048-4096. Usually larger than output dimension.

InputDimension

Gets the input dimension expected by this projector.

int InputDimension { get; }

Property Value

int

OutputDimension

Gets the output dimension produced by this projector.

int OutputDimension { get; }

Property Value

int

Remarks

Typical values: 128-2048. SimCLR uses 128, MoCo uses 128, BYOL uses 256.

ParameterCount

Gets the total number of trainable parameters.

int ParameterCount { get; }

Property Value

int

Methods

Backward(Tensor<T>)

Performs the backward pass through the projector.

Tensor<T> Backward(Tensor<T> gradients)

Parameters

gradients Tensor<T>

The gradients from the loss with respect to projector output.

Returns

Tensor<T>

The gradients with respect to projector input (for encoder backprop).

ClearGradients()

Clears accumulated gradients.

void ClearGradients()

GetParameterGradients()

Gets the gradients computed during the last backward pass.

Vector<T> GetParameterGradients()

Returns

Vector<T>

A vector containing gradients for all parameters.

GetParameters()

Gets all trainable parameters of the projector.

Vector<T> GetParameters()

Returns

Vector<T>

A vector containing all parameters.

Project(Tensor<T>)

Projects encoder output to the SSL embedding space.

Tensor<T> Project(Tensor<T> input)

Parameters

input Tensor<T>

The encoder output tensor.

Returns

Tensor<T>

The projected embedding tensor.

Remarks

For Beginners: This transforms encoder features into a lower-dimensional space where the SSL loss is computed. The projection helps separate the pretraining objective from the learned representations.

Reset()

Resets the projector state (clears any internal buffers).

void Reset()

SetParameters(Vector<T>)

Sets the parameters of the projector.

void SetParameters(Vector<T> parameters)

Parameters

parameters Vector<T>

The parameter vector to load.

SetTrainingMode(bool)

Sets training or evaluation mode.

void SetTrainingMode(bool isTraining)

Parameters

isTraining bool

True for training mode, false for evaluation.