Class TensorPool

Namespace

AiDotNet.JitCompiler.Memory

Assembly

AiDotNet.dll

Provides efficient tensor memory pooling to reduce allocations and GC pressure during JIT execution.

public class TensorPool : IDisposable

Inheritance

object

TensorPool

Implements

IDisposable

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

For Beginners: This is like a "rental service" for tensor memory.

Creating and destroying large tensors repeatedly is expensive because:

1. Memory allocation takes time
2. Garbage collection causes pauses
3. Memory fragmentation reduces performance

The TensorPool keeps frequently-used tensor buffers around and recycles them:

1. When you need a tensor, borrow one from the pool
2. When you're done, return it to the pool
3. Next time someone needs a tensor of that size, they get your recycled one

This dramatically improves performance for repeated computations like training loops.

Constructors

TensorPool()

Creates a new tensor pool with default settings.

public TensorPool()

TensorPool(int, int)

Creates a new tensor pool with custom settings.

public TensorPool(int maxPoolSizePerShape, int maxElementsToPool)

Parameters

maxPoolSizePerShape int

Maximum number of tensors to keep per shape.

maxElementsToPool int

Maximum total elements in a tensor to pool (larger tensors won't be pooled).

Methods

Clear()

Clears all pooled buffers, allowing them to be garbage collected.

public void Clear()

Dispose()

Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources.

public void Dispose()

GetStats()

Gets statistics about the current pool state.

public TensorPoolStats GetStats()

Returns

TensorPoolStats

Pool statistics including buffer counts and estimated memory usage.

Rent<T>(int)

Rents a tensor buffer of the specified size.

public T[] Rent<T>(int totalElements)

Parameters

totalElements int

Total number of elements needed.

Returns

T[]

An array that may be recycled from the pool or newly allocated.

Type Parameters

T

The element type of the tensor.

Remarks

For Beginners: Gets a buffer for your tensor data.

The buffer might be recycled from a previous tensor, so it may contain old data. You should initialize or overwrite all values before using the tensor.

Example: var buffer = pool.Rent<float>(1000); // Use buffer for computation... pool.Return(buffer);

Return<T>(T[])

Returns a tensor buffer to the pool for reuse.

public void Return<T>(T[] buffer)

Parameters

buffer T[]

The buffer to return.

Type Parameters

T

The element type of the tensor.

Remarks

For Beginners: Gives back a buffer you're done using.

After returning a buffer, you must not use it anymore! The buffer might be given to someone else immediately.

Important:

• Always return buffers you rented
• Never use a buffer after returning it
• Don't return buffers you didn't rent from this pool