Interface IRLDataLoader<T>

Namespace

AiDotNet.Interfaces

Assembly

AiDotNet.dll

Interface for data loaders that provide experience data for reinforcement learning.

public interface IRLDataLoader<T> : IDataLoader<T>, IResettable, ICountable, IBatchIterable<Experience<T, Vector<T>, Vector<T>>>

Type Parameters

T

The numeric type used for calculations, typically float or double.

Inherited Members

IDataLoader<T>.Name

IDataLoader<T>.Description

IDataLoader<T>.IsLoaded

IDataLoader<T>.LoadAsync(CancellationToken)

IDataLoader<T>.Unload()

IResettable.Reset()

ICountable.TotalCount

ICountable.CurrentIndex

ICountable.BatchCount

ICountable.CurrentBatchIndex

ICountable.Progress

IBatchIterable<Experience<T, Vector<T>, Vector<T>>>.BatchSize

IBatchIterable<Experience<T, Vector<T>, Vector<T>>>.HasNext

IBatchIterable<Experience<T, Vector<T>, Vector<T>>>.GetNextBatch()

IBatchIterable<Experience<T, Vector<T>, Vector<T>>>.TryGetNextBatch(out Experience<T, Vector<T>, Vector<T>>)

IBatchIterable<Experience<T, Vector<T>, Vector<T>>>.GetBatches(int?, bool, bool, int?)

IBatchIterable<Experience<T, Vector<T>, Vector<T>>>.GetBatchesAsync(int?, bool, bool, int?, int, CancellationToken)

Extension Methods

ParallelBatchLoaderExtensions.WithParallelLoading<TBatch>(IBatchIterable<TBatch>, Func<int[], TBatch>, int, int?, int?)

DataPipelineExtensions.ToPipeline<TBatch>(IBatchIterable<TBatch>)

DataLoaderExtensions.CreateBatchesAsync<TBatch>(IBatchIterable<TBatch>, int?, int)

DataLoaderExtensions.CreateBatches<TBatch>(IBatchIterable<TBatch>, int?)

Remarks

This interface is for reinforcement learning scenarios where an agent interacts with an environment to collect experience data. The loader manages: - Environment interactions (stepping through episodes) - Experience collection and storage - Replay buffer management for batch sampling

For Beginners: Reinforcement learning is learning through trial and error.

How it works:

• An agent takes actions in an environment
• The environment returns rewards and new states
• The agent learns to maximize total rewards

Example: Game Playing

• Environment: The game (e.g., CartPole, Atari, Chess)
• State: What the agent sees (game screen, piece positions)
• Action: What the agent does (move left, jump, place piece)
• Reward: Score or outcome (+1 for winning, -1 for losing)

This data loader:

• Runs episodes in the environment
• Collects experience tuples (state, action, reward, next_state, done)
• Stores them in a replay buffer for training
• Provides batches of experiences for learning

Properties

CurrentEpisode

Gets the current episode number (0-indexed).

int CurrentEpisode { get; }

Property Value

int

Environment

Gets the environment that the agent interacts with.

IEnvironment<T> Environment { get; }

Property Value

IEnvironment<T>

Episodes

Gets the total number of episodes to run during training.

int Episodes { get; }

Property Value

int

MaxStepsPerEpisode

Gets the maximum number of steps per episode (prevents infinite episodes).

int MaxStepsPerEpisode { get; }

Property Value

int

MinExperiencesBeforeTraining

Gets the minimum number of experiences required before training can begin.

int MinExperiencesBeforeTraining { get; }

Property Value

int

Remarks

For Beginners: We need some experiences before we can learn from random samples. This ensures the replay buffer has enough diverse experiences for effective learning.

ReplayBuffer

Gets the replay buffer used for storing and sampling experiences.

IReplayBuffer<T, Vector<T>, Vector<T>> ReplayBuffer { get; }

Property Value

IReplayBuffer<T, Vector<T>, Vector<T>>

TotalSteps

Gets the total number of steps taken across all episodes.

int TotalSteps { get; }

Property Value

int

Verbose

Gets whether to print training progress to console.

bool Verbose { get; }

Property Value

bool

Methods

AddExperience(Experience<T, Vector<T>, Vector<T>>)

Adds an experience to the replay buffer.

void AddExperience(Experience<T, Vector<T>, Vector<T>> experience)

Parameters

experience Experience<T, Vector<T>, Vector<T>>

The experience to add.

CanTrain(int)

Checks if there are enough experiences to begin training.

bool CanTrain(int batchSize)

Parameters

batchSize int

The desired batch size for training.

Returns

bool

True if training can proceed, false if more experiences are needed.

ResetTraining()

Resets the data loader state (clears buffer, resets counters).

void ResetTraining()

RunEpisode(IRLAgent<T>?)

Runs a single episode and collects experiences.

EpisodeResult<T> RunEpisode(IRLAgent<T>? agent = null)

Parameters

agent IRLAgent<T>

Optional agent to use for action selection. If null, uses random actions.

Returns

EpisodeResult<T>

Episode result containing total reward, steps, and whether it was successful.

RunEpisodes(int, IRLAgent<T>?)

Runs multiple episodes and collects experiences.

IReadOnlyList<EpisodeResult<T>> RunEpisodes(int numEpisodes, IRLAgent<T>? agent = null)

Parameters

numEpisodes int

Number of episodes to run.

agent IRLAgent<T>

Optional agent to use for action selection.

Returns

IReadOnlyList<EpisodeResult<T>>

List of episode results.

SampleBatch(int)

Samples a batch of experiences from the replay buffer.

IReadOnlyList<Experience<T, Vector<T>, Vector<T>>> SampleBatch(int batchSize)

Parameters

batchSize int

Number of experiences to sample.

Returns

IReadOnlyList<Experience<T, Vector<T>, Vector<T>>>

List of sampled experiences for training.

Remarks

For Beginners: Instead of learning from experiences in order (which can cause issues), we randomly sample from past experiences. This makes learning more stable.

SetSeed(int)

Sets the random seed for reproducible training.

void SetSeed(int seed)

Parameters

seed int

Random seed value.