Class RobustnessStats<T>

Namespace

AiDotNet.Models

Assembly

AiDotNet.dll

Represents adversarial robustness diagnostics aggregated over a dataset.

public sealed class RobustnessStats<T>

Type Parameters

T

The numeric type used for calculations (e.g., float, double).

Inheritance

object

RobustnessStats<T>

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

This container is designed to integrate with the existing AiDotNet evaluation pipeline by living alongside ErrorStats<T> and PredictionStats<T> within DataSetStats<T, TInput, TOutput>. It stores metrics related to model robustness against adversarial attacks and certified defenses.

For Beginners: This stores summary robustness metrics (like accuracy under attack) for an entire dataset, helping you understand how well your model resists adversarial perturbations.

Key concepts:

• Clean Accuracy: How accurate the model is on unmodified inputs
• Adversarial Accuracy: How accurate the model is when inputs are perturbed by attacks
• Certified Accuracy: The fraction of samples with provably correct predictions within a perturbation radius
• Attack Success Rate: How often an attacker can fool the model
• Average Perturbation Size: How much inputs need to be changed to fool the model

Properties

AdditionalMetrics

Gets a dictionary of additional robustness metrics.

public Dictionary<string, T> AdditionalMetrics { get; }

Property Value

Dictionary<string, T>

Remarks

For Beginners: This stores any extra metrics that don't fit the standard properties, allowing for extensibility without changing the class structure.

AdversarialAccuracy

Gets the accuracy of the model on adversarially perturbed inputs.

public double AdversarialAccuracy { get; set; }

Property Value

double

Remarks

For Beginners: This measures how often the model still makes correct predictions when inputs have been modified by an adversarial attack. Lower values mean the model is more vulnerable to attacks.

AttackSuccessRate

Gets the fraction of inputs for which the attack successfully fooled the model.

public double AttackSuccessRate { get; set; }

Property Value

double

Remarks

For Beginners: This is 1 - AdversarialAccuracy for correctly classified clean inputs. A high attack success rate means the model is easy to fool.

AttackType

Gets the type of attack used for adversarial robustness evaluation.

public string AttackType { get; set; }

Property Value

string

Remarks

For Beginners: Different attacks have different strengths: - FGSM: Fast but weak - PGD: Slower but stronger - C&W: Slowest but often finds smallest perturbations - AutoAttack: Ensemble of attacks for reliable evaluation

AverageCertifiedRadius

Gets the average certified robustness radius across samples.

public double AverageCertifiedRadius { get; set; }

Property Value

double

Remarks

For Beginners: This is the average "safety zone" around inputs where predictions are guaranteed to stay the same. Larger radii mean stronger certified robustness.

AveragePerturbationSize

Gets the average size of perturbations needed to create successful adversarial examples.

public double AveragePerturbationSize { get; set; }

Property Value

double

Remarks

For Beginners: This measures how much the input needs to be changed on average to fool the model. Larger values suggest the model is more robust (harder to fool with small changes).

CertifiedAccuracy

Gets the certified accuracy at the specified perturbation radius.

public double CertifiedAccuracy { get; set; }

Property Value

double

Remarks

For Beginners: This is the fraction of predictions that are mathematically guaranteed to be correct even if the input is perturbed within a certain radius. Unlike adversarial accuracy (which tests specific attacks), certified accuracy provides provable guarantees against ALL possible perturbations within the radius.

CleanAccuracy

Gets the accuracy of the model on clean (unperturbed) inputs.

public double CleanAccuracy { get; set; }

Property Value

double

Remarks

For Beginners: This is the normal accuracy you would measure without any attacks. It serves as a baseline to compare against adversarial accuracy.

EvaluationEpsilon

Gets the perturbation radius (epsilon) used for robustness evaluation.

public double EvaluationEpsilon { get; set; }

Property Value

double

Remarks

For Beginners: This is the maximum allowed perturbation size used when evaluating robustness. For image data, this is often around 8/255 ≈ 0.031 for L-infinity attacks (imperceptible pixel changes).

IsEvaluated

Gets or sets whether robustness evaluation has been performed.

public bool IsEvaluated { get; set; }

Property Value

bool

NormType

Gets the norm type used for measuring perturbation size (e.g., "L2", "Linf").

public string NormType { get; set; }

Property Value

string

RobustnessScore

Gets a combined robustness score (0-1) that balances clean and adversarial performance.

public double RobustnessScore { get; set; }

Property Value

double

Remarks

For Beginners: This is a single number that summarizes overall robustness. Higher values indicate better robustness. The default formula is: (CleanAccuracy + AdversarialAccuracy) / 2

Methods

Empty()

Creates an empty RobustnessStats<T> instance.

public static RobustnessStats<T> Empty()

Returns

RobustnessStats<T>