Table of Contents

Class MoCoV2<T>

Namespace
AiDotNet.SelfSupervisedLearning
Assembly
AiDotNet.dll

MoCo v2: Improved Baselines with Momentum Contrastive Learning.

public class MoCoV2<T> : MoCo<T>, ISSLMethod<T>

Type Parameters

T

The numeric type used for computations.

Inheritance
MoCo<T>
MoCoV2<T>
Implements
Inherited Members

Remarks

For Beginners: MoCo v2 improves on MoCo v1 by incorporating ideas from SimCLR: an MLP projection head and stronger augmentations. This combines MoCo's memory efficiency with SimCLR's representation quality improvements.

Key improvements over MoCo v1:

  • MLP projection head: 2-layer MLP instead of linear (from SimCLR)
  • Stronger augmentations: Added blur and more color distortion (from SimCLR)
  • Cosine learning rate schedule: Better training dynamics

Result: MoCo v2 matches SimCLR performance with much smaller batch sizes (256 vs 4096-8192).

Reference: Chen et al., "Improved Baselines with Momentum Contrastive Learning" (arXiv 2020)

Constructors

MoCoV2(INeuralNetwork<T>, IMomentumEncoder<T>, IProjectorHead<T>, IProjectorHead<T>, int, SSLConfig?)

Initializes a new instance of the MoCoV2 class.

public MoCoV2(INeuralNetwork<T> encoder, IMomentumEncoder<T> momentumEncoder, IProjectorHead<T> projector, IProjectorHead<T> momentumProjector, int embeddingDim = 128, SSLConfig? config = null)

Parameters

encoder INeuralNetwork<T>

The online encoder network.

momentumEncoder IMomentumEncoder<T>

The momentum encoder.

projector IProjectorHead<T>

The MLP projection head for online encoder.

momentumProjector IProjectorHead<T>

The MLP projection head for momentum encoder.

embeddingDim int

Dimension of the output embeddings.

config SSLConfig

Optional SSL configuration.

Properties

Name

Gets the name of this SSL method.

public override string Name { get; }

Property Value

string

Remarks

Examples: "SimCLR", "MoCo v2", "BYOL", "DINO", "MAE"

Methods

Create(INeuralNetwork<T>, Func<INeuralNetwork<T>, INeuralNetwork<T>>, int, int, int, int)

Creates a MoCo v2 instance with default configuration.

public static MoCoV2<T> Create(INeuralNetwork<T> encoder, Func<INeuralNetwork<T>, INeuralNetwork<T>> createEncoderCopy, int encoderOutputDim, int projectionDim = 128, int hiddenDim = 2048, int queueSize = 65536)

Parameters

encoder INeuralNetwork<T>

The backbone encoder.

createEncoderCopy Func<INeuralNetwork<T>, INeuralNetwork<T>>

Function to create a copy of the encoder for momentum.

encoderOutputDim int

Output dimension of the encoder.

projectionDim int

Dimension of the projection space (default: 128).

hiddenDim int

Hidden dimension of the projector MLP (default: 2048).

queueSize int

Size of the memory queue (default: 65536).

Returns

MoCoV2<T>

A configured MoCo v2 instance.

Edit this page