Table of Contents

Class SPLADE<T>

Namespace
AiDotNet.NeuralNetworks
Assembly
AiDotNet.dll

SPLADE (Sparse Lexical and Expansion Model) neural network implementation. Maps text to a high-dimensional sparse vector in the vocabulary space.

public class SPLADE<T> : TransformerEmbeddingNetwork<T>, INeuralNetworkModel<T>, INeuralNetwork<T>, IFullModel<T, Tensor<T>, Tensor<T>>, IModel<Tensor<T>, Tensor<T>, ModelMetadata<T>>, IModelSerializer, ICheckpointableModel, IParameterizable<T, Tensor<T>, Tensor<T>>, IFeatureAware, IFeatureImportance<T>, ICloneable<IFullModel<T, Tensor<T>, Tensor<T>>>, IGradientComputable<T, Tensor<T>, Tensor<T>>, IJitCompilable<T>, IInterpretableModel<T>, IInputGradientComputable<T>, IDisposable, IEmbeddingModel<T>

Type Parameters

T

The numeric type used for calculations (typically float or double).

Inheritance
SPLADE<T>
Implements
IFullModel<T, Tensor<T>, Tensor<T>>
IModel<Tensor<T>, Tensor<T>, ModelMetadata<T>>
IParameterizable<T, Tensor<T>, Tensor<T>>
ICloneable<IFullModel<T, Tensor<T>, Tensor<T>>>
IGradientComputable<T, Tensor<T>, Tensor<T>>
Inherited Members
Extension Methods

Remarks

SPLADE is a sparse retrieval model that learns to represent documents and queries as sparse vectors over the vocabulary. It uses a log-saturation effect and sparsity regularization (e.g., FLOPs or L1) to learn lexical expansion and term importance.

For Beginners: Imagine a dictionary with 30,000 words. For every sentence, SPLADE creates a giant list of 30,000 numbers, but almost all of them are zero. It only puts numbers next to the words that are actually important to the meaning.

The "Expansion" part is the most interesting: if you say "The smartphone is fast," SPLADE might automatically put a number next to the word "Apple" or "Android" in its dictionary, even if you didn't say them. This helps it find relevant documents that use different words.

Constructors

SPLADE(NeuralNetworkArchitecture<T>, ITokenizer?, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>>?, int, int, int, int, int, int, ILossFunction<T>?, double)

Initializes a new instance of the SPLADE model.

public SPLADE(NeuralNetworkArchitecture<T> architecture, ITokenizer? tokenizer = null, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>>? optimizer = null, int vocabSize = 30522, int embeddingDimension = 768, int maxSequenceLength = 512, int numLayers = 12, int numHeads = 12, int feedForwardDim = 3072, ILossFunction<T>? lossFunction = null, double maxGradNorm = 1)

Parameters

architecture NeuralNetworkArchitecture<T>
tokenizer ITokenizer
optimizer IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>>
vocabSize int
embeddingDimension int
maxSequenceLength int
numLayers int
numHeads int
feedForwardDim int
lossFunction ILossFunction<T>
maxGradNorm double

Methods

CreateNewInstance()

Creates a new instance of the same type as this neural network.

protected override IFullModel<T, Tensor<T>, Tensor<T>> CreateNewInstance()

Returns

IFullModel<T, Tensor<T>, Tensor<T>>

A new instance of the same neural network type.

Remarks

For Beginners: This creates a blank version of the same type of neural network.

It's used internally by methods like DeepCopy and Clone to create the right type of network before copying the data into it.

DeserializeNetworkSpecificData(BinaryReader)

Deserializes network-specific data that was not covered by the general deserialization process.

protected override void DeserializeNetworkSpecificData(BinaryReader reader)

Parameters

reader BinaryReader

The BinaryReader to read the data from.

Remarks

This method is called at the end of the general deserialization process to allow derived classes to read any additional data specific to their implementation.

For Beginners: Continuing the suitcase analogy, this is like unpacking that special compartment. After the main deserialization method has unpacked the common items (layers, parameters), this method allows each specific type of neural network to unpack its own unique items that were stored during serialization.

Embed(string)

Encodes text into a high-dimensional sparse lexical representation.

public override Vector<T> Embed(string text)

Parameters

text string

Returns

Vector<T>

EmbedAsync(string)

Asynchronously embeds a single text string into a vector representation.

public override Task<Vector<T>> EmbedAsync(string text)

Parameters

text string

The text to embed.

Returns

Task<Vector<T>>

A task representing the async operation, with the resulting vector.

EmbedBatchAsync(IEnumerable<string>)

Asynchronously embeds multiple text strings into vector representations in a single batch operation.

public override Task<Matrix<T>> EmbedBatchAsync(IEnumerable<string> texts)

Parameters

texts IEnumerable<string>

The collection of texts to embed.

Returns

Task<Matrix<T>>

A task representing the async operation, with the resulting matrix.

GetModelMetadata()

Retrieves detailed metadata about the SPLADE configuration.

public override ModelMetadata<T> GetModelMetadata()

Returns

ModelMetadata<T>

InitializeLayers()

Configures the transformer backbone and the ReLU-based expansion head for SPLADE.

protected override void InitializeLayers()

SerializeNetworkSpecificData(BinaryWriter)

Serializes network-specific data that is not covered by the general serialization process.

protected override void SerializeNetworkSpecificData(BinaryWriter writer)

Parameters

writer BinaryWriter

The BinaryWriter to write the data to.

Remarks

This method is called at the end of the general serialization process to allow derived classes to write any additional data specific to their implementation.

For Beginners: Think of this as packing a special compartment in your suitcase. While the main serialization method packs the common items (layers, parameters), this method allows each specific type of neural network to pack its own unique items that other networks might not have.

Edit this page