Class ActivationPool<T>

Namespace

AiDotNet.Diffusion.Memory

Assembly

AiDotNet.dll

Memory pool for tensor activations during diffusion model forward/backward passes.

public class ActivationPool<T> : IDisposable

Type Parameters

T

The numeric type used for calculations.

Inheritance

object

ActivationPool<T>

Implements

IDisposable

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

Diffusion models process large tensors through many layers, creating significant memory pressure from intermediate activations. This pool reduces allocations by recycling tensor buffers.

For Beginners: When running diffusion models, temporary data (activations) is created at each layer.

Without pooling:

• Layer 1 creates activation A (allocate new memory)
• Layer 2 creates activation B (allocate new memory)
• Layer 1's activation A becomes garbage (GC must clean up)
• This creates memory pressure and GC pauses

With pooling:

• Layer 1 borrows a buffer from the pool
• Layer 1 returns the buffer when done
• Layer 2 reuses the same buffer
• No garbage, no GC pauses, faster inference

Usage:

using var pool = new ActivationPool<float>(maxMemoryMB: 2048);

// During forward pass
var activation = pool.Rent(new[] { 1, 256, 64, 64 });
// ... use activation ...
pool.Return(activation);

Constructors

ActivationPool(long)

Initializes a new activation pool with specified memory limit.

public ActivationPool(long maxMemoryMB = 4096)

Parameters

maxMemoryMB long

Maximum memory in megabytes (default: 4096 MB).

Remarks

The pool will evict older tensors when approaching the memory limit. Set this based on your available GPU/CPU memory minus what the model weights use.

Properties

Stats

Statistics about pool usage.

public ActivationPoolStats Stats { get; }

Property Value

ActivationPoolStats

Methods

Clear()

Clears all pooled tensors and resets memory accounting.

public void Clear()

Dispose()

Disposes the pool and releases all tensors.

public void Dispose()

GetMemoryUsage()

Gets current memory usage statistics.

public long GetMemoryUsage()

Returns

long

Current memory usage in bytes.

Rent(int[])

Rents a tensor from the pool or creates a new one.

public Tensor<T> Rent(int[] shape)

Parameters

shape int[]

Desired tensor shape.

Returns

Tensor<T>

A tensor ready for use (contents may be uninitialized).

Remarks

Important: The returned tensor may contain data from previous uses. Always initialize the tensor before reading from it.

Return(Tensor<T>)

Returns a tensor to the pool for reuse.

public void Return(Tensor<T> tensor)

Parameters

tensor Tensor<T>

The tensor to return.

Remarks

Important: Do not use the tensor after returning it to the pool. The pool may immediately give it to another caller.