Class NumericalStabilityHelper
Provides numerical stability utilities for safe mathematical operations in machine learning.
public static class NumericalStabilityHelper- Inheritance
-
NumericalStabilityHelper
- Inherited Members
Remarks
For Beginners: Machine learning algorithms often deal with very small or very large numbers, which can cause numerical issues like: - Division by zero - Log of zero or negative numbers - NaN (Not a Number) values appearing in calculations - Infinity values from overflow
This helper provides safe versions of common operations that avoid these problems.
Fields
DefaultEpsilon
Default epsilon value for numerical stability (1e-7 for float precision).
public const double DefaultEpsilon = 1E-07Field Value
LargeEpsilon
Larger epsilon for less sensitive operations (1e-5).
public const double LargeEpsilon = 1E-05Field Value
SmallEpsilon
Smaller epsilon for double precision operations (1e-15).
public const double SmallEpsilon = 1E-15Field Value
Methods
AssertFinite<T>(Tensor<T>, string)
Asserts that a tensor contains only finite values.
public static void AssertFinite<T>(Tensor<T> tensor, string paramName = "tensor")Parameters
tensorTensor<T>The tensor to check.
paramNamestringName of the parameter for the exception message.
Type Parameters
TThe numeric type.
Exceptions
- ArgumentException
Thrown if the tensor contains non-finite values.
AssertFinite<T>(Vector<T>, string)
Asserts that a vector contains only finite values.
public static void AssertFinite<T>(Vector<T> vector, string paramName = "vector")Parameters
vectorVector<T>The vector to check.
paramNamestringName of the parameter for the exception message.
Type Parameters
TThe numeric type.
Exceptions
- ArgumentException
Thrown if the vector contains non-finite values.
AssertFinite<T>(T, string)
Asserts that a value is finite, throwing if not.
public static void AssertFinite<T>(T value, string paramName = "value")Parameters
valueTThe value to check.
paramNamestringName of the parameter for the exception message.
Type Parameters
TThe numeric type.
Exceptions
- ArgumentException
Thrown if the value is not finite.
ClampProbability<T>(T, double)
Clamps a value to valid probability range [epsilon, 1-epsilon].
public static T ClampProbability<T>(T probability, double epsilon = 1E-07)Parameters
probabilityTThe probability value to clamp.
epsilondoubleSmall value for bounds. Defaults to 1e-7.
Returns
- T
The clamped probability.
Type Parameters
TThe numeric type.
Remarks
For Beginners: Probabilities should be between 0 and 1, but for numerical stability (especially when taking log of probabilities), we clamp to [epsilon, 1-epsilon] to avoid log(0) and log(1) issues.
ContainsInfinity<T>(Tensor<T>)
Checks if a tensor contains any infinite values.
public static bool ContainsInfinity<T>(Tensor<T> tensor)Parameters
tensorTensor<T>The tensor to check.
Returns
- bool
True if any element is infinite.
Type Parameters
TThe numeric type.
ContainsInfinity<T>(Vector<T>)
Checks if a vector contains any infinite values.
public static bool ContainsInfinity<T>(Vector<T> vector)Parameters
vectorVector<T>The vector to check.
Returns
- bool
True if any element is infinite.
Type Parameters
TThe numeric type.
ContainsNaN<T>(Tensor<T>)
Checks if a tensor contains any NaN values.
public static bool ContainsNaN<T>(Tensor<T> tensor)Parameters
tensorTensor<T>The tensor to check.
Returns
- bool
True if any element is NaN.
Type Parameters
TThe numeric type.
ContainsNaN<T>(Vector<T>)
Checks if a vector contains any NaN values.
public static bool ContainsNaN<T>(Vector<T> vector)Parameters
vectorVector<T>The vector to check.
Returns
- bool
True if any element is NaN.
Type Parameters
TThe numeric type.
ContainsNonFinite<T>(Tensor<T>)
Checks if a tensor contains any non-finite values (NaN or infinite).
public static bool ContainsNonFinite<T>(Tensor<T> tensor)Parameters
tensorTensor<T>The tensor to check.
Returns
- bool
True if any element is non-finite.
Type Parameters
TThe numeric type.
ContainsNonFinite<T>(Vector<T>)
Checks if a vector contains any non-finite values (NaN or infinite).
public static bool ContainsNonFinite<T>(Vector<T> vector)Parameters
vectorVector<T>The vector to check.
Returns
- bool
True if any element is non-finite.
Type Parameters
TThe numeric type.
CountInfinity<T>(Vector<T>)
Counts the number of infinite values in a vector.
public static int CountInfinity<T>(Vector<T> vector)Parameters
vectorVector<T>The vector to check.
Returns
- int
The count of infinite values.
Type Parameters
TThe numeric type.
CountNaN<T>(Vector<T>)
Counts the number of NaN values in a vector.
public static int CountNaN<T>(Vector<T> vector)Parameters
vectorVector<T>The vector to check.
Returns
- int
The count of NaN values.
Type Parameters
TThe numeric type.
GetEpsilon<T>(double?)
Gets a type-appropriate epsilon value for the numeric type T.
public static T GetEpsilon<T>(double? epsilon = null)Parameters
epsilondouble?Optional custom epsilon. If null, uses type-appropriate default.
Returns
- T
The epsilon value converted to type T.
Type Parameters
TThe numeric type.
IsFinite<T>(T)
Checks if a value is finite (not NaN and not infinite).
public static bool IsFinite<T>(T value)Parameters
valueTThe value to check.
Returns
- bool
True if the value is finite.
Type Parameters
TThe numeric type.
IsInfinity<T>(T)
Checks if a value is infinite (positive or negative infinity).
public static bool IsInfinity<T>(T value)Parameters
valueTThe value to check.
Returns
- bool
True if the value is infinite.
Type Parameters
TThe numeric type.
IsNaN<T>(T)
Checks if a value is NaN (Not a Number).
public static bool IsNaN<T>(T value)Parameters
valueTThe value to check.
Returns
- bool
True if the value is NaN.
Type Parameters
TThe numeric type.
ReplaceInfinity<T>(Vector<T>?, T?)
Replaces infinite values in a vector with a specified replacement value.
public static Vector<T>? ReplaceInfinity<T>(Vector<T>? vector, T? replacement = default)Parameters
vectorVector<T>The vector to process.
replacementTThe value to replace infinity with (defaults to zero).
Returns
- Vector<T>
A new vector with infinite values replaced.
Type Parameters
TThe numeric type.
ReplaceNaN<T>(Vector<T>?, T?)
Replaces NaN values in a vector with a specified replacement value.
public static Vector<T>? ReplaceNaN<T>(Vector<T>? vector, T? replacement = default)Parameters
vectorVector<T>The vector to process.
replacementTThe value to replace NaN with (defaults to zero).
Returns
- Vector<T>
A new vector with NaN values replaced.
Type Parameters
TThe numeric type.
ReplaceNonFinite<T>(Vector<T>?, T?)
Replaces all non-finite values (NaN and infinity) in a vector.
public static Vector<T>? ReplaceNonFinite<T>(Vector<T>? vector, T? replacement = default)Parameters
vectorVector<T>The vector to process.
replacementTThe value to replace non-finite values with (defaults to zero).
Returns
- Vector<T>
A new vector with non-finite values replaced.
Type Parameters
TThe numeric type.
SafeDiv<T>(T, T, double)
Performs safe division, avoiding division by zero.
public static T SafeDiv<T>(T numerator, T denominator, double epsilon = 1E-07)Parameters
numeratorTThe numerator.
denominatorTThe denominator.
epsilondoubleSmall value to add to denominator for stability. Defaults to 1e-7.
Returns
- T
numerator / (denominator + epsilon) if denominator is near zero, else numerator / denominator.
Type Parameters
TThe numeric type.
Remarks
For Beginners: Division by zero results in infinity or NaN. This method adds a tiny value to very small denominators to prevent this while minimally affecting the result.
SafeLogProbability<T>(T, double)
Computes safe log of a probability (clamps first, then takes log).
public static T SafeLogProbability<T>(T probability, double epsilon = 1E-07)Parameters
probabilityTThe probability value.
epsilondoubleSmall value for stability. Defaults to 1e-7.
Returns
- T
log(clamp(probability, epsilon, 1-epsilon))
Type Parameters
TThe numeric type.
SafeLog<T>(T, double)
Computes the natural logarithm safely, avoiding log(0) and log(negative).
public static T SafeLog<T>(T value, double epsilon = 1E-07)Parameters
valueTThe value to compute log of.
epsilondoubleSmall value to add for numerical stability. Defaults to 1e-7.
Returns
- T
log(max(value, epsilon))
Type Parameters
TThe numeric type.
Remarks
For Beginners: The logarithm of zero is negative infinity, and log of negative numbers is undefined. This method ensures we always compute log of a small positive number at minimum, preventing NaN or -Infinity in your calculations.
SafeSqrt<T>(T, double)
Computes square root safely, ensuring non-negative input.
public static T SafeSqrt<T>(T value, double epsilon = 1E-07)Parameters
valueTThe value to compute square root of.
epsilondoubleSmall value to ensure positive input. Defaults to 1e-7.
Returns
- T
sqrt(max(value, epsilon))
Type Parameters
TThe numeric type.
StableLogSoftmax<T>(Vector<T>?)
Computes log-softmax with numerical stability.
public static Vector<T>? StableLogSoftmax<T>(Vector<T>? logits)Parameters
logitsVector<T>The input logits.
Returns
- Vector<T>
Log-softmax values.
Type Parameters
TThe numeric type.
Remarks
For Beginners: Log-softmax is log(softmax(x)), which is more numerically stable than computing softmax first and then taking the log. It's commonly used in cross-entropy loss calculations.
StableSoftmax<T>(Vector<T>?)
Computes softmax with numerical stability using the log-sum-exp trick.
public static Vector<T>? StableSoftmax<T>(Vector<T>? logits)Parameters
logitsVector<T>The input logits.
Returns
- Vector<T>
Softmax probabilities.
Type Parameters
TThe numeric type.
Remarks
For Beginners: Softmax converts a vector of numbers into probabilities. The standard formula exp(x_i) / sum(exp(x_j)) can overflow for large values. This implementation subtracts the maximum value first to prevent overflow.