Table of Contents

Interface IRLAgent<T>

Namespace
AiDotNet.Interfaces
Assembly
AiDotNet.dll

Marker interface for reinforcement learning agents that integrate with AiModelBuilder.

public interface IRLAgent<T> : IFullModel<T, Vector<T>, Vector<T>>, IModel<Vector<T>, Vector<T>, ModelMetadata<T>>, IModelSerializer, ICheckpointableModel, IParameterizable<T, Vector<T>, Vector<T>>, IFeatureAware, IFeatureImportance<T>, ICloneable<IFullModel<T, Vector<T>, Vector<T>>>, IGradientComputable<T, Vector<T>, Vector<T>>, IJitCompilable<T>

Type Parameters

T

The numeric type used for calculations.

Inherited Members
Extension Methods

Remarks

This interface extends IFullModel to ensure RL agents integrate seamlessly with AiDotNet's existing architecture. RL agents are models where: - TInput = Tensor<T> (state observations, though often flattened to Vector in practice) - TOutput = Vector<T> (actions)

For Beginners: An RL agent is just a special kind of model that learns through interaction with an environment. By implementing IFullModel, RL agents work with all of AiDotNet's existing infrastructure: - They can be saved and loaded - They work with the AiModelBuilder pattern - They support serialization, cloning, etc.

The key difference is how they're trained:

  • Regular models: trained on fixed datasets (x, y)
  • RL agents: trained by interacting with environments and getting rewards

Methods

GetMetrics()

Gets current training metrics.

Dictionary<string, T> GetMetrics()

Returns

Dictionary<string, T>

Dictionary of metric names to values.

ResetEpisode()

Resets episode-specific state (if any).

void ResetEpisode()

SelectAction(Vector<T>, bool)

Selects an action given the current state observation.

Vector<T> SelectAction(Vector<T> state, bool explore = true)

Parameters

state Vector<T>

The current state observation.

explore bool

Whether to use exploration (epsilon-greedy, etc.).

Returns

Vector<T>

Action as a Vector (one-hot for discrete, continuous values for continuous action spaces).

Remarks

For Beginners: This is how the agent decides what to do in a given situation. During training, it might explore (try random things), but during evaluation it uses its learned policy.

StoreExperience(Vector<T>, Vector<T>, T, Vector<T>, bool)

Stores an experience tuple for later learning.

void StoreExperience(Vector<T> state, Vector<T> action, T reward, Vector<T> nextState, bool done)

Parameters

state Vector<T>

The state before taking action.

action Vector<T>

The action taken.

reward T

The reward received.

nextState Vector<T>

The state after taking action.

done bool

Whether the episode terminated.

Remarks

For Beginners: RL agents learn from experiences. This stores one experience (state, action, reward, next state) for the agent to learn from later.

Train()

Performs one training step using stored experiences.

T Train()

Returns

T

Training loss for monitoring.

Remarks

For Beginners: This is where the agent actually learns from its experiences. It looks at what happened (stored experiences) and updates its strategy to get better rewards.

Edit this page