Class CategoricalCrossEntropyLoss<T>

Namespace

AiDotNet.LossFunctions

Assembly

AiDotNet.dll

Implements the Categorical Cross Entropy loss function for multi-class classification.

public class CategoricalCrossEntropyLoss<T> : LossFunctionBase<T>, ILossFunction<T>

Type Parameters

T

The numeric type used for calculations (e.g., float, double).

Inheritance

object

LossFunctionBase<T>

CategoricalCrossEntropyLoss<T>

Implements

ILossFunction<T>

Inherited Members

LossFunctionBase<T>.NumOps

LossFunctionBase<T>.Engine

LossFunctionBase<T>.ValidateVectorLengths(Vector<T>, Vector<T>)

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Extension Methods

LossFunctionExtensions.ComputeGradient<T>(ILossFunction<T>, Tensor<T>, Tensor<T>)

LossFunctionExtensions.ComputeLoss<T>(ILossFunction<T>, Tensor<T>, Tensor<T>)

Remarks

For Beginners: Categorical Cross Entropy is used for multi-class classification problems, where you need to assign inputs to one of several categories (like classifying images as dog, cat, bird, etc.).

It measures how well the predicted probability distribution matches the actual distribution of classes.

The formula is: CCE = -(1/n) * ?[?(actual_j * log(predicted_j))]

Where:

• actual_j is usually a one-hot encoded vector (1 for the correct class, 0 for others)
• predicted_j is the predicted probability for each class (typically from a softmax output)
• The inner sum is over all classes, and the outer sum is over all samples

Key properties:

• Predicted values should be probabilities (between 0 and 1) that sum to 1 across classes
• It heavily penalizes confident incorrect predictions
• It's the standard loss function for multi-class neural network classifiers
• Often used together with the softmax activation function in the output layer

This loss function is ideal when your model needs to choose one option from multiple possibilities.

Constructors

CategoricalCrossEntropyLoss()

Initializes a new instance of the CategoricalCrossEntropyLoss class.

public CategoricalCrossEntropyLoss()

Methods

CalculateDerivative(Vector<T>, Vector<T>)

Calculates the derivative of the Categorical Cross Entropy loss function.

public override Vector<T> CalculateDerivative(Vector<T> predicted, Vector<T> actual)

Parameters

predicted Vector<T>

The predicted values (probabilities that sum to 1 across categories).

actual Vector<T>

The actual (target) values (typically one-hot encoded).

Returns

Vector<T>

A vector containing the derivatives of CCE for each prediction.

CalculateLoss(Vector<T>, Vector<T>)

Calculates the Categorical Cross Entropy loss between predicted and actual values.

public override T CalculateLoss(Vector<T> predicted, Vector<T> actual)

Parameters

predicted Vector<T>

The predicted values (probabilities that sum to 1 across categories).

actual Vector<T>

The actual (target) values (typically one-hot encoded).

Returns

T

The categorical cross entropy loss value.

CalculateLossAndGradientGpu(IGpuTensor<T>, IGpuTensor<T>)

Calculates both Categorical Cross Entropy loss and gradient on GPU in a single efficient pass.

public override (T Loss, IGpuTensor<T> Gradient) CalculateLossAndGradientGpu(IGpuTensor<T> predicted, IGpuTensor<T> actual)

Parameters

predicted IGpuTensor<T>

The predicted GPU tensor from the model.

actual IGpuTensor<T>

The actual (target) GPU tensor.

Returns

(T Loss, IGpuTensor<T> Gradient)

A tuple containing the loss value and gradient tensor.