Table of Contents

Class ContinualEvaluationResult<T>

Namespace
AiDotNet.ContinualLearning.Results
Assembly
AiDotNet.dll

Comprehensive evaluation result across all learned tasks.

public class ContinualEvaluationResult<T>

Type Parameters

T

The numeric type used for calculations.

Inheritance
ContinualEvaluationResult<T>
Inherited Members

Remarks

For Beginners: This class provides a complete picture of how well the model performs across all tasks it has learned. It includes key metrics for measuring continual learning effectiveness.

Key Metrics Explained:

  • Average Accuracy: Mean accuracy across all tasks (higher is better)
  • Backward Transfer: How learning new tasks affects old task performance (positive = improvement, negative = forgetting)
  • Forward Transfer: How old knowledge helps with new tasks (positive = positive transfer)
  • Forgetting: Maximum accuracy drop on any previous task (lower is better)

Reference: Lopez-Paz and Ranzato "Gradient Episodic Memory for Continual Learning" (2017)

Constructors

ContinualEvaluationResult(Vector<T>, Vector<T>, T, T, T, T, TimeSpan)

Initializes a new instance of the ContinualEvaluationResult<T> class.

public ContinualEvaluationResult(Vector<T> taskAccuracies, Vector<T> taskLosses, T averageAccuracy, T averageLoss, T backwardTransfer, T forwardTransfer, TimeSpan evaluationTime)

Parameters

taskAccuracies Vector<T>
taskLosses Vector<T>
averageAccuracy T
averageLoss T
backwardTransfer T
forwardTransfer T
evaluationTime TimeSpan

Properties

AccuracyMatrix

Gets the accuracy matrix R where R[i,j] is accuracy on task j after learning task i.

public Matrix<T>? AccuracyMatrix { get; init; }

Property Value

Matrix<T>

Remarks

This matrix enables detailed analysis of learning dynamics over time.

AverageAccuracy

Gets the average accuracy across all tasks.

public T AverageAccuracy { get; }

Property Value

T

AverageLoss

Gets the average loss across all tasks.

public T AverageLoss { get; }

Property Value

T

BackwardTransfer

Gets the backward transfer metric.

public T BackwardTransfer { get; }

Property Value

T

Remarks

Definition: BWT = (1/T-1) * sum_{i=1}^{T-1} (R_{T,i} - R_{i,i})

Where R_{j,i} is the accuracy on task i after learning task j.

Negative BWT indicates forgetting. Positive BWT indicates that learning later tasks improved performance on earlier tasks.

EvaluationTime

Gets the time taken for evaluation.

public TimeSpan EvaluationTime { get; }

Property Value

TimeSpan

ForwardTransfer

Gets the forward transfer metric.

public T ForwardTransfer { get; }

Property Value

T

Remarks

Definition: FWT = (1/T-1) * sum_{i=2}^{T} (R_{i-1,i} - R_{0,i})

Where R_{j,i} is the accuracy on task i after learning task j, and R_{0,i} is the accuracy on task i before any training.

Positive FWT indicates that learning previous tasks helped with new tasks.

Intransigence

Gets the intransigence metric (inability to learn new tasks).

public T? Intransigence { get; init; }

Property Value

T

Remarks

Measures how much worse the model is at learning new tasks compared to a model trained only on that task. Higher values indicate difficulty adapting.

LearningEfficiency

Gets the learning curve efficiency (area under the learning curve).

public T? LearningEfficiency { get; init; }

Property Value

T

MaxForgetting

Gets the maximum forgetting observed on any task.

public T? MaxForgetting { get; init; }

Property Value

T

Remarks

MaxForgetting = max_i (max_j(R_{j,i}) - R_{T,i})

This is the worst-case accuracy drop on any single task.

PerTaskResults

Gets the per-task evaluation results.

public IReadOnlyList<TaskEvaluationResult<T>>? PerTaskResults { get; init; }

Property Value

IReadOnlyList<TaskEvaluationResult<T>>

StabilityPlasticityRatio

Gets the stability-plasticity ratio.

public T? StabilityPlasticityRatio { get; init; }

Property Value

T

Remarks

Measures the trade-off between retaining old knowledge (stability) and learning new information (plasticity). Closer to 1 indicates good balance.

TaskAccuracies

Gets the accuracy on each task.

public Vector<T> TaskAccuracies { get; }

Property Value

Vector<T>

TaskCount

Gets the total number of tasks evaluated.

public int TaskCount { get; }

Property Value

int

TaskLosses

Gets the loss on each task.

public Vector<T> TaskLosses { get; }

Property Value

Vector<T>

Methods

GenerateReport()

Generates a detailed report of the evaluation results.

public string GenerateReport()

Returns

string

ToString()

Returns a string representation of the evaluation result.

public override string ToString()

Returns

string
Edit this page