Class TensorPool<T>

Namespace

AiDotNet.Memory

Assembly

AiDotNet.dll

A high-performance, thread-safe memory pool for reusing tensors during neural network operations. Reduces memory allocations and garbage collection pressure by pooling tensor buffers.

public class TensorPool<T> : IDisposable

Type Parameters

T

The numeric element type of tensors in this pool (e.g., float, double).

Inheritance

object

TensorPool<T>

Implements

IDisposable

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

The tensor pool maintains buckets of pre-allocated tensors grouped by shape. When a tensor is requested via Rent(int[]), the pool returns an existing tensor from the appropriate bucket if available, otherwise allocates a new one.

When tensors are returned via Return(Tensor<T>), they are cleared and added back to the pool for future reuse, up to the configured memory limits.

Basic usage example:

using var pool = new TensorPool<float>(maxPoolSizeMB: 256);
// Rent a tensor from the pool
var tensor = pool.Rent(new[] { 32, 784 });
// Use the tensor for computations...
// Return it to the pool when done
pool.Return(tensor);

For automatic lifetime management, use RentPooled(int[]) which returns a PooledTensor<T> that automatically returns itself when disposed:

using var pooled = pool.RentPooled(new[] { 32, 784 });
// Use pooled.Tensor for computations...
// Tensor is automatically returned when 'using' block exits

Constructors

TensorPool()

Initializes a new instance of the TensorPool<T> class with default options. Default max pool size is 256 MB.

public TensorPool()

TensorPool(PoolingOptions)

Initializes a new instance of the TensorPool<T> class with the specified options.

public TensorPool(PoolingOptions options)

Parameters

options PoolingOptions

The pooling configuration options. If null, default options are used.

TensorPool(int)

Initializes a new instance of the TensorPool<T> class with the specified maximum pool size.

public TensorPool(int maxPoolSizeMB)

Parameters

maxPoolSizeMB int

The maximum memory size for the pool in megabytes.

Properties

CurrentMemoryBytes

Gets the current memory usage of all pooled tensors, in bytes.

public long CurrentMemoryBytes { get; }

Property Value

long

CurrentPoolSizeBytes

Gets the current size of the pool in bytes. Alias for CurrentMemoryBytes.

public long CurrentPoolSizeBytes { get; }

Property Value

long

MaxPoolSizeBytes

Gets the maximum allowed memory size for the pool, in bytes.

public long MaxPoolSizeBytes { get; }

Property Value

long

Options

Gets the pooling options configured for this pool.

public PoolingOptions Options { get; }

Property Value

PoolingOptions

TotalPooledTensors

Gets the total number of tensors currently held in the pool across all buckets.

public int TotalPooledTensors { get; }

Property Value

int

Methods

Clear()

Removes all tensors and memory from the pool and resets memory tracking.

public void Clear()

Remarks

After calling Clear, all pooled tensors and memory become eligible for garbage collection. New tensors and memory can still be rented and returned after clearing.

Dispose()

Disposes the tensor pool and releases all pooled tensors. After disposal, the pool cannot be used.

public void Dispose()

GetStatistics()

Gets current statistics about the pool's state.

public PoolStatistics GetStatistics()

Returns

PoolStatistics

A PoolStatistics object containing pool metrics.

Examples

var stats = pool.GetStatistics();
Console.WriteLine($"Pool utilization: {stats.MemoryUtilizationPercent:F1}%");
Console.WriteLine($"Tensors in pool: {stats.PooledTensorCount}");

Rent(int[])

Rents a tensor with the specified shape from the pool. If a matching tensor is available in the pool, it is returned after being cleared. Otherwise, a new tensor is allocated.

public Tensor<T> Rent(int[] shape)

Parameters

shape int[]

The shape of the tensor to rent (e.g., [32, 784] for a 2D tensor).

Returns

Tensor<T>

A tensor with the requested shape, either from the pool or newly allocated.

Exceptions

ObjectDisposedException

Thrown if the pool has been disposed.

ArgumentException

Thrown if shape is null, empty, or contains non-positive dimensions.

RentMemory(int)

Rents raw memory from the pool for lower-level operations. Returns an IMemoryOwner that automatically returns the memory when disposed.

public IMemoryOwner<T> RentMemory(int elementCount)

Parameters

elementCount int

The number of elements to allocate.

Returns

IMemoryOwner<T>

An IMemoryOwner wrapping the pooled memory.

Examples

using var memory = pool.RentMemory(1024);
var span = memory.Memory.Span;
// Use span... memory is automatically returned when block exits

Exceptions

ObjectDisposedException

Thrown if the pool has been disposed.

RentPooled(int[])

Rents a tensor wrapped in a PooledTensor<T> for automatic pool return on disposal.

public PooledTensor<T> RentPooled(int[] shape)

Parameters

shape int[]

The shape of the tensor to rent.

Returns

PooledTensor<T>

A pooled tensor wrapper that returns the tensor to the pool when disposed.

Examples

using var pooled = pool.RentPooled(new[] { 32, 32 });
var tensor = pooled.Tensor;
// Use tensor... it's automatically returned when block exits

Exceptions

ObjectDisposedException

Thrown if the pool has been disposed.

Return(Tensor<T>)

Returns a tensor to the pool for future reuse. The tensor is cleared and added to the appropriate shape bucket if memory limits allow.

public void Return(Tensor<T> tensor)

Parameters

tensor Tensor<T>

The tensor to return to the pool.

Remarks

If the tensor is null, too large for pooling, or the pool is at capacity, the tensor is not pooled and will be garbage collected normally.

Exceptions

ObjectDisposedException

Thrown if the pool has been disposed.