Table of Contents

Class MetaEvaluationResult<T>

Namespace
AiDotNet.Models.Results
Assembly
AiDotNet.dll

Results from evaluating a meta-learner across multiple tasks.

public class MetaEvaluationResult<T>

Type Parameters

T

The numeric type used for calculations (e.g., float, double, decimal).

Inheritance
MetaEvaluationResult<T>
Inherited Members

Remarks

This class aggregates evaluation metrics from multiple tasks to assess meta-learning performance. It uses the existing BasicStats infrastructure to provide comprehensive statistical analysis of how well the meta-learner adapts to new tasks.

For Beginners: Meta-learning evaluation tests how well your model can quickly learn new tasks.

This result tells you:

  • Average accuracy: How well the model performs on new tasks after quick adaptation
  • Consistency: How much performance varies across different tasks (standard deviation)
  • Confidence: Statistical confidence intervals for the results
  • Per-task details: Individual task results for deep analysis

For example, if you're doing 5-way 1-shot classification:

  • You sample many test tasks (e.g., 100 tasks)
  • For each task, the model sees 1 example per class and must classify new examples
  • This result aggregates accuracy and loss across all those tasks
  • Higher mean accuracy and lower standard deviation indicate better meta-learning

Constructors

MetaEvaluationResult(Vector<T>, Vector<T>, TimeSpan, Dictionary<string, T>?)

Initializes a new instance with task results and calculates all statistics.

public MetaEvaluationResult(Vector<T> taskAccuracies, Vector<T> taskLosses, TimeSpan evaluationTime, Dictionary<string, T>? additionalMetrics = null)

Parameters

taskAccuracies Vector<T>

Accuracy values from all evaluated tasks.

taskLosses Vector<T>

Loss values from all evaluated tasks.

evaluationTime TimeSpan

Time taken for evaluation.

additionalMetrics Dictionary<string, T>

Optional algorithm-specific metrics.

Remarks

This constructor follows the established pattern of calculating statistics in the constructor. It uses the existing BasicStats infrastructure to compute comprehensive statistics from the raw per-task values, ensuring consistency with other result classes in the codebase.

For Beginners: This constructor takes your raw task results and automatically calculates all the statistical summaries (mean, std dev, etc.).

You typically call this from meta-learning evaluation code after collecting results from many tasks. The constructor does all the statistical work for you.

Exceptions

ArgumentNullException

Thrown when taskAccuracies or taskLosses is null.

ArgumentException

Thrown when vectors have different lengths or are empty.

Properties

AccuracyStats

Gets comprehensive statistics for accuracy across all evaluated tasks.

public BasicStats<T> AccuracyStats { get; }

Property Value

BasicStats<T>

BasicStats containing mean, standard deviation, median, quartiles, and distribution metrics for accuracy values across tasks. Accuracy is measured on query sets after adaptation.

Remarks

For Beginners: This contains all the statistical measures for accuracy: - Mean: The average accuracy across all tasks - StandardDeviation: How much accuracy varies between tasks - Median: The middle accuracy value (less affected by outliers) - Min/Max: The worst and best task accuracies

Lower standard deviation means the model consistently performs well across different tasks.

AdditionalMetrics

Gets algorithm-specific metrics that don't fit standard categories.

public Dictionary<string, T> AdditionalMetrics { get; }

Property Value

Dictionary<string, T>

A dictionary of custom metrics with generic T values.

Remarks

For Production: Common additional metrics include: - "support_accuracy": Accuracy on support sets (should be very high) - "adaptation_time_ms": Average time per task adaptation - "convergence_rate": How quickly inner loop converged - "gradient_magnitude": Size of adaptation gradients

EvaluationTime

Gets the total time taken for evaluation.

public TimeSpan EvaluationTime { get; }

Property Value

TimeSpan

The elapsed time for evaluating all tasks.

LossStats

Gets comprehensive statistics for loss across all evaluated tasks.

public BasicStats<T> LossStats { get; }

Property Value

BasicStats<T>

BasicStats containing mean, standard deviation, median, quartiles, and distribution metrics for loss values across tasks. Loss is measured on query sets after adaptation.

NumTasks

Gets the number of tasks used for evaluation.

public int NumTasks { get; }

Property Value

int

The total count of tasks sampled and evaluated.

Remarks

For Beginners: More tasks give more reliable statistics.

Typical values:

  • Research papers: 600-1000 tasks for final evaluation
  • Quick validation: 100-200 tasks
  • Debugging: 10-50 tasks

PerTaskAccuracies

Gets the individual accuracy values for each evaluated task.

public Vector<T> PerTaskAccuracies { get; }

Property Value

Vector<T>

A vector containing one accuracy value per task, in the order tasks were evaluated.

Remarks

For Production: Use this for: - Detailed analysis of task-level performance - Identifying problematic task types - Creating histograms or other visualizations - Statistical hypothesis testing

PerTaskLosses

Gets the individual loss values for each evaluated task.

public Vector<T> PerTaskLosses { get; }

Property Value

Vector<T>

A vector containing one loss value per task, in the order tasks were evaluated.

Methods

GenerateReport()

Generates a formatted summary report of the evaluation results.

public string GenerateReport()

Returns

string

A human-readable string containing key evaluation metrics.

Remarks

For Beginners: This creates a text summary you can print or log to quickly see how well your meta-learner performed.

It includes the most important metrics:

  • Mean accuracy ± standard deviation
  • 95% confidence interval
  • Loss statistics
  • Number of tasks evaluated
  • Evaluation time

GetAccuracyConfidenceInterval()

Calculates the 95% confidence interval for mean accuracy.

public (T Lower, T Upper) GetAccuracyConfidenceInterval()

Returns

(T Accuracy, T Loss)

A tuple containing the lower and upper bounds of the confidence interval.

Remarks

Uses the standard error of the mean with a z-score of 1.96 (95% confidence level). Assumes the distribution of task accuracies is approximately normal, which is reasonable for large sample sizes (Central Limit Theorem).

For Beginners: The confidence interval tells you the range where the true average accuracy likely falls.

For example, if you get (0.82, 0.88) with mean accuracy 0.85:

  • You can be 95% confident the true mean accuracy is between 82% and 88%
  • Narrower intervals mean more precise estimates
  • More tasks (larger NumTasks) give narrower intervals

GetLossConfidenceInterval()

Calculates the 95% confidence interval for mean loss.

public (T Lower, T Upper) GetLossConfidenceInterval()

Returns

(T Accuracy, T Loss)

A tuple containing the lower and upper bounds of the confidence interval.

Edit this page