Table of Contents

Class SparsePruningCompression<T>

Namespace
AiDotNet.ModelCompression
Assembly
AiDotNet.dll

Implements sparse pruning compression by zeroing out small-magnitude weights.

public class SparsePruningCompression<T> : ModelCompressionBase<T>, IModelCompressionStrategy<T>

Type Parameters

T

The numeric type used for calculations (e.g., float, double).

Inheritance
SparsePruningCompression<T>
Implements
Inherited Members

Remarks

Sparse pruning removes weights below a certain threshold, setting them to zero. This creates sparsity in the model which can be exploited for efficient storage using sparse matrix formats (only non-zero values and their indices are stored).

For Beginners: Sparse pruning is like cleaning out your closet.

The idea:

  • Many neural network weights are very small (close to zero)
  • These tiny weights contribute little to the output
  • We can set them to exactly zero without much accuracy loss
  • Only store the non-zero weights and their positions

How it works:

  1. Calculate a threshold (e.g., smallest 90% of weights by magnitude)
  2. Set all weights below threshold to zero
  3. Store only non-zero values with their indices

Benefits:

  • Can achieve 90%+ sparsity (90% zeros) with minimal accuracy loss
  • Sparse storage is very efficient (only store ~10% of weights)
  • Works well combined with quantization or clustering

Example:

  • Original: [0.001, 0.5, -0.002, 0.8, 0.003, -0.7]
  • After 50% pruning: [0, 0.5, 0, 0.8, 0, -0.7]
  • Sparse storage: values=[0.5, 0.8, -0.7], indices=[1, 3, 5]

Constructors

SparsePruningCompression(double, double, bool)

Initializes a new instance of the SparsePruningCompression class.

public SparsePruningCompression(double sparsityTarget = 0.9, double minMagnitudeThreshold = 0, bool useGlobalThreshold = true)

Parameters

sparsityTarget double

Target sparsity level (0.0 to 1.0, default: 0.9 = 90% zeros).

minMagnitudeThreshold double

Minimum magnitude threshold (default: 0 = use sparsity target).

useGlobalThreshold bool

Whether to use a global threshold or per-layer (default: true).

Remarks

For Beginners: These parameters control pruning aggressiveness:

  • sparsityTarget: What fraction of weights to set to zero

    • 0.5 = 50% zeros (mild pruning)
    • 0.9 = 90% zeros (aggressive pruning, common choice)
    • 0.95 = 95% zeros (very aggressive)

  • minMagnitudeThreshold: Absolute threshold (overrides sparsity target if > 0)

    • 0.01 means all weights with |w| < 0.01 become zero
    • Useful when you know a good threshold for your model

  • useGlobalThreshold: Apply same threshold to all weights

    • true = find one threshold for entire model
    • false = find separate threshold for each layer (preserves layer balance)

Methods

Compress(Vector<T>)

Compresses weights by pruning small-magnitude values and storing in sparse format.

public override (Vector<T> compressedWeights, ICompressionMetadata<T> metadata) Compress(Vector<T> weights)

Parameters

weights Vector<T>

The original model weights.

Returns

(Vector<T> compressedWeights, ICompressionMetadata<T> metadata)

Compressed sparse representation and metadata.

Decompress(Vector<T>, ICompressionMetadata<T>)

Decompresses sparse weights back to dense format.

public override Vector<T> Decompress(Vector<T> compressedWeights, ICompressionMetadata<T> metadata)

Parameters

compressedWeights Vector<T>

The non-zero weight values.

metadata ICompressionMetadata<T>

The metadata containing indices and original length.

Returns

Vector<T>

The reconstructed dense weights (with zeros filled in).

GetCompressedSize(Vector<T>, ICompressionMetadata<T>)

Gets the compressed size including sparse values and indices.

public override long GetCompressedSize(Vector<T> compressedWeights, ICompressionMetadata<T> metadata)

Parameters

compressedWeights Vector<T>
metadata ICompressionMetadata<T>

Returns

long
Edit this page