Table of Contents

Class SSLConfig

Namespace
AiDotNet.SelfSupervisedLearning
Assembly
AiDotNet.dll

Unified configuration for self-supervised learning with industry-standard defaults.

public class SSLConfig
Inheritance
SSLConfig
Inherited Members

Remarks

For Beginners: Self-supervised learning (SSL) learns useful representations from unlabeled data. This configuration controls how SSL pretraining works, including the method to use, batch size, training epochs, and method-specific settings.

Key features:

  • Automatic method selection (defaults to SimCLR)
  • Industry-standard defaults that work well out-of-the-box
  • Method-specific settings for fine-tuning (MoCo, BYOL, DINO, MAE)
  • Evaluation settings for linear probe and k-NN evaluation

Example - Simple usage with defaults:

var result = builder
    .ConfigureModel(encoder)
    .ConfigureSelfSupervisedLearning()  // Uses SimCLR with defaults
    .Build(unlabeledData);

Example - Custom configuration:

builder.ConfigureSelfSupervisedLearning(config =>
{
    config.Method = SSLMethodType.MoCo;
    config.PretrainingEpochs = 200;
    config.BatchSize = 256;
    config.MoCo = new MoCoConfig { QueueSize = 65536 };
});

Constructors

SSLConfig()

Creates a new SSL configuration with industry-standard defaults.

public SSLConfig()

Properties

BYOL

Gets or sets BYOL-specific configuration.

public BYOLConfig? BYOL { get; set; }

Property Value

BYOLConfig

BarlowTwins

Gets or sets Barlow Twins-specific configuration.

public BarlowTwinsConfig? BarlowTwins { get; set; }

Property Value

BarlowTwinsConfig

BatchSize

Gets or sets the batch size for pretraining.

public int? BatchSize { get; set; }

Property Value

int?

Remarks

Default: 256

Larger batch sizes generally help contrastive methods (SimCLR benefits from 4096-8192).

CheckpointFrequency

Gets or sets the checkpoint save frequency (epochs).

public int? CheckpointFrequency { get; set; }

Property Value

int?

Remarks

Default: 20

Set to 0 to disable automatic checkpointing.

CheckpointPath

Gets or sets the path to save checkpoints.

public string? CheckpointPath { get; set; }

Property Value

string

DINO

Gets or sets DINO-specific configuration.

public DINOConfig? DINO { get; set; }

Property Value

DINOConfig

Distributed

Gets or sets the distributed training configuration.

public SSLDistributedConfig? Distributed { get; set; }

Property Value

SSLDistributedConfig

Remarks

Default: null (single-GPU/CPU training)

When set, enables distributed data parallel (DDP) training across multiple GPUs/nodes.

For Beginners: Distributed training allows you to use multiple GPUs to train faster. Each GPU processes a different batch, and gradients are averaged across all GPUs. This is especially important for SSL methods like SimCLR that benefit from very large effective batch sizes.

EnableKNNEvaluation

Gets or sets whether to run k-NN evaluation.

public bool? EnableKNNEvaluation { get; set; }

Property Value

bool?

Remarks

Default: true

k-NN evaluation is faster than linear and doesn't require training.

EnableLinearEvaluation

Gets or sets whether to run linear evaluation during/after pretraining.

public bool? EnableLinearEvaluation { get; set; }

Property Value

bool?

Remarks

Default: true

Linear evaluation trains a linear classifier on frozen features to measure representation quality.

KNNNeighbors

Gets or sets the number of neighbors for k-NN evaluation.

public int? KNNNeighbors { get; set; }

Property Value

int?

Remarks

Default: 20

LearningRate

Gets or sets the base learning rate.

public double? LearningRate { get; set; }

Property Value

double?

Remarks

Default: 0.3 (with linear scaling rule)

The effective learning rate is: base_lr * batch_size / 256

LinearEvaluationFrequency

Gets or sets the frequency of linear evaluation (epochs).

public int? LinearEvaluationFrequency { get; set; }

Property Value

int?

Remarks

Default: 10

Set to 0 to only evaluate at the end of training.

MAE

Gets or sets MAE-specific configuration.

public MAEConfig? MAE { get; set; }

Property Value

MAEConfig

Method

Gets or sets the SSL method to use.

public SSLMethodType? Method { get; set; }

Property Value

SSLMethodType?

Remarks

Default: SSLMethodType.SimCLR

If null, SimCLR is used as the default method.

MoCo

Gets or sets MoCo-specific configuration.

public MoCoConfig? MoCo { get; set; }

Property Value

MoCoConfig

OptimizerType

Gets or sets the optimizer type for SSL training.

public SSLOptimizerType? OptimizerType { get; set; }

Property Value

SSLOptimizerType?

Remarks

Default: SSLOptimizerType.LARS

LARS is recommended for large batch SSL training. LAMB is better for transformer-based encoders.

PretrainingEpochs

Gets or sets the number of pretraining epochs.

public int? PretrainingEpochs { get; set; }

Property Value

int?

Remarks

Default: 100

SSL typically requires more epochs than supervised learning. Research papers often use 200-1000 epochs.

ProjectorHiddenDimension

Gets or sets the hidden dimension of the projection head MLP.

public int? ProjectorHiddenDimension { get; set; }

Property Value

int?

Remarks

Default: 2048

ProjectorLayers

Gets or sets the number of layers in the projection head.

public int? ProjectorLayers { get; set; }

Property Value

int?

Remarks

Default: 2

Common values: 2 (SimCLR, MoCo), 3 (BYOL with predictor).

ProjectorOutputDimension

Gets or sets the output dimension of the projection head.

public int? ProjectorOutputDimension { get; set; }

Property Value

int?

Remarks

Default: 128

Common values: 128 (SimCLR, MoCo), 256 (BYOL), 2048 (Barlow Twins).

Seed

Gets or sets the random seed for reproducibility.

public int? Seed { get; set; }

Property Value

int?

Temperature

Gets or sets the temperature parameter for contrastive loss.

public double? Temperature { get; set; }

Property Value

double?

Remarks

Default: 0.07

Lower temperature makes the distribution sharper (harder negatives).

Common values: 0.07 (MoCo), 0.1 (SimCLR), 0.5 (some variants).

UseCosineDecay

Gets or sets whether to use cosine learning rate decay.

public bool? UseCosineDecay { get; set; }

Property Value

bool?

Remarks

Default: true

Cosine decay is standard for SSL pretraining.

UseTemperatureScheduling

Gets or sets whether to use temperature scheduling.

public bool? UseTemperatureScheduling { get; set; }

Property Value

bool?

Remarks

Default: false

Some methods benefit from scheduling temperature during training.

WarmupEpochs

Gets or sets the number of warmup epochs.

public int? WarmupEpochs { get; set; }

Property Value

int?

Remarks

Default: 10

Linear warmup helps stabilize early training.

WeightDecay

Gets or sets the weight decay (L2 regularization).

public double? WeightDecay { get; set; }

Property Value

double?

Remarks

Default: 1e-4

Methods

GetConfiguration()

Gets the configuration as a dictionary for logging or serialization.

public IDictionary<string, object> GetConfiguration()

Returns

IDictionary<string, object>

A dictionary containing all configuration values.

Edit this page