Class CrossEntropyLoss<T>

Namespace

AiDotNet.LossFunctions

Assembly

AiDotNet.dll

Implements the Cross Entropy loss function for multi-class classification problems.

public class CrossEntropyLoss<T> : LossFunctionBase<T>, ILossFunction<T>

Type Parameters

T

The numeric type used for calculations (e.g., float, double).

Inheritance

object

LossFunctionBase<T>

CrossEntropyLoss<T>

Implements

ILossFunction<T>

Inherited Members

LossFunctionBase<T>.NumOps

LossFunctionBase<T>.Engine

LossFunctionBase<T>.ValidateVectorLengths(Vector<T>, Vector<T>)

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Extension Methods

LossFunctionExtensions.ComputeGradient<T>(ILossFunction<T>, Tensor<T>, Tensor<T>)

LossFunctionExtensions.ComputeLoss<T>(ILossFunction<T>, Tensor<T>, Tensor<T>)

Remarks

For Beginners: Cross-Entropy loss measures how different two probability distributions are. It's commonly used for classification problems where the model outputs probabilities.

The formula is: -?(actual_i * log(predicted_i))

Key properties:

• Lower values indicate that the predicted distribution is closer to the actual distribution
• It encourages the model to be confident about correct predictions
• It heavily penalizes confident but incorrect predictions
• It's particularly suited for training classifiers

This loss function is often used in conjunction with the softmax activation function in the output layer for multi-class classification problems.

Constructors

CrossEntropyLoss()

Initializes a new instance of the CrossEntropyLoss class.

public CrossEntropyLoss()

Methods

CalculateDerivative(Vector<T>, Vector<T>)

Calculates the derivative of the Cross-Entropy loss function.

public override Vector<T> CalculateDerivative(Vector<T> predicted, Vector<T> actual)

Parameters

predicted Vector<T>

The predicted probability distribution.

actual Vector<T>

The actual (target) probability distribution.

Returns

Vector<T>

A vector containing the derivative of Cross-Entropy loss for each element.

CalculateLoss(Vector<T>, Vector<T>)

Calculates the Cross-Entropy loss between predicted and actual probability distributions.

public override T CalculateLoss(Vector<T> predicted, Vector<T> actual)

Parameters

predicted Vector<T>

The predicted probability distribution.

actual Vector<T>

The actual (target) probability distribution.

Returns

T

The Cross-Entropy loss value.

CalculateLossAndGradientGpu(IGpuTensor<T>, IGpuTensor<T>)

Calculates both Cross-Entropy loss and gradient on GPU in a single efficient pass.

public override (T Loss, IGpuTensor<T> Gradient) CalculateLossAndGradientGpu(IGpuTensor<T> predicted, IGpuTensor<T> actual)

Parameters

predicted IGpuTensor<T>

The predicted GPU tensor (probability distribution).

actual IGpuTensor<T>

The actual (target) GPU tensor.

Returns

(T Loss, IGpuTensor<T> Gradient)

A tuple containing the loss value and gradient tensor.