Table of Contents

Class HuffmanEncodingCompression<T>

Namespace
AiDotNet.ModelCompression
Assembly
AiDotNet.dll

Implements Huffman encoding compression for model weights using variable-length encoding.

public class HuffmanEncodingCompression<T> : ModelCompressionBase<T>, IModelCompressionStrategy<T>

Type Parameters

T

The numeric type used for calculations (e.g., float, double).

Inheritance
HuffmanEncodingCompression<T>
Implements
Inherited Members

Remarks

Huffman encoding is a lossless compression technique that assigns shorter codes to more frequent values and longer codes to less frequent values. This is particularly effective when combined with weight clustering, where cluster indices have non-uniform frequency distributions.

For Beginners: Huffman encoding is like creating custom abbreviations.

Imagine you're taking notes in a lecture:

  • Words you hear often (like "the", "and", "is") you abbreviate with single letters
  • Rare words you write out in full
  • This makes your notes much shorter overall

For neural networks:

  • Some weight values appear much more frequently than others
  • Frequent values get short binary codes (like "01")
  • Rare values get longer codes (like "110101")
  • Since frequent values appear often, using short codes saves a lot of space

The magic is that Huffman encoding is "lossless":

  • You can perfectly reconstruct the original values
  • No accuracy is lost (unlike clustering which is lossy)
  • It's often combined with clustering for even better compression

Example:

  • Value appearing 1000 times: code "1" (1 bit each = 1000 bits total)
  • Value appearing 10 times: code "01001" (5 bits each = 50 bits total)
  • Total: 1050 bits instead of possibly much more with fixed-length codes

Constructors

HuffmanEncodingCompression(int)

Initializes a new instance of the HuffmanEncodingCompression class.

public HuffmanEncodingCompression(int precision = 6)

Parameters

precision int

The number of decimal places to round to before encoding (default: 6).

Remarks

For Beginners: Precision controls how many decimal places to keep.

  • Higher precision = more accurate but less compression

    • precision=8: 0.12345678 (very precise, more unique values, less compression)

  • Lower precision = more compression but slightly less accurate

    • precision=4: 0.1235 (rounded, fewer unique values, better compression)

The default of 6 decimal places is usually a good balance.

Why round at all?

  • Without rounding, you might have millions of unique values
  • Huffman encoding works best when many values are the same
  • Rounding creates more duplicate values, improving compression

Methods

Compress(Vector<T>)

Compresses weights using Huffman encoding.

public override (Vector<T> compressedWeights, ICompressionMetadata<T> metadata) Compress(Vector<T> weights)

Parameters

weights Vector<T>

The original model weights.

Returns

(Vector<T> compressedWeights, ICompressionMetadata<T> metadata)

Compressed weights and metadata containing the Huffman tree and encoding table.

Decompress(Vector<T>, ICompressionMetadata<T>)

Decompresses weights by decoding the Huffman-encoded bit stream.

public override Vector<T> Decompress(Vector<T> compressedWeights, ICompressionMetadata<T> metadata)

Parameters

compressedWeights Vector<T>

The compressed weights (encoded as bytes).

metadata ICompressionMetadata<T>

The metadata containing the Huffman tree.

Returns

Vector<T>

The decompressed weights.

GetCompressedSize(Vector<T>, ICompressionMetadata<T>)

Gets the compressed size including the Huffman tree and encoded bits.

public override long GetCompressedSize(Vector<T> compressedWeights, ICompressionMetadata<T> metadata)

Parameters

compressedWeights Vector<T>
metadata ICompressionMetadata<T>

Returns

long
Edit this page