Class ParameterAnalyzer<T>

Namespace

AiDotNet.DistributedTraining

Assembly

AiDotNet.dll

Analyzes model parameters and creates optimized groupings for distributed communication.

public class ParameterAnalyzer<T>

Type Parameters

T

The numeric type

Inheritance

object

ParameterAnalyzer<T>

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

For Beginners: Think of ParameterAnalyzer as a smart packing assistant. When shipping items, you don't want to send thousands of tiny packages - it's inefficient! Instead, you group small items together into larger boxes.

Similarly, when communicating parameters across GPUs: - Sending many small parameter arrays is slow (lots of communication overhead) - Grouping small parameters together reduces the number of messages - This analyzer figures out the best way to group parameters for efficiency

For example, instead of sending 1000 separate bias vectors (each with 1 parameter), we might group them into 10 larger chunks (each with 100 parameters).

Constructors

ParameterAnalyzer(int, int)

Creates a new parameter analyzer with the specified settings.

public ParameterAnalyzer(int minimumGroupSize = 1024, int worldSize = 1)

Parameters

minimumGroupSize int

Minimum size for a parameter group (smaller groups will be merged)

worldSize int

Number of processes in the distributed group

Remarks

For Beginners: This creates the analyzer that will figure out how to group parameters. You tell it: - minimumGroupSize: The smallest acceptable group size (smaller groups get merged) - worldSize: How many processes are sharing the work (affects optimal group sizes)

Methods

AnalyzeForDistribution(Vector<T>)

Analyzes parameters and creates groups optimized for even distribution across processes.

public List<ParameterAnalyzer<T>.ParameterGroup> AnalyzeForDistribution(Vector<T> parameters)

Parameters

parameters Vector<T>

The parameter vector to analyze

Returns

List<ParameterAnalyzer<T>.ParameterGroup>

A list of parameter groups optimized for distribution

Remarks

For Beginners: When distributing work across multiple processes, we want each process to get roughly the same amount of work. This method creates groups that divide evenly.

For example, if you have 10,000 parameters and 4 processes: - Each process should get ~2,500 parameters - We create groups sized to divide evenly by 4

AnalyzeModel<TInput, TOutput>(IFullModel<T, TInput, TOutput>)

Analyzes a model's parameters and creates optimized groupings.

public List<ParameterAnalyzer<T>.ParameterGroup> AnalyzeModel<TInput, TOutput>(IFullModel<T, TInput, TOutput> model)

Parameters

model IFullModel<T, TInput, TOutput>

The model to analyze

Returns

List<ParameterAnalyzer<T>.ParameterGroup>

A list of optimized parameter groups

Type Parameters

TInput

The input type of the model

TOutput

The output type of the model

Remarks

For Beginners: This method looks at all the parameters in your model and decides how to group them for efficient communication. It returns a list of ParameterGroups, each representing parameters that should be sent together.

The analyzer: 1. Identifies natural parameter boundaries (e.g., weights vs biases) 2. Merges small groups that are below the minimum size 3. Ensures groups are aligned with process boundaries for even distribution

AnalyzeParameters(Vector<T>)

Analyzes a parameter vector and creates optimized groupings.

public List<ParameterAnalyzer<T>.ParameterGroup> AnalyzeParameters(Vector<T> parameters)

Parameters

parameters Vector<T>

The parameter vector to analyze

Returns

List<ParameterAnalyzer<T>.ParameterGroup>

A list of optimized parameter groups

Remarks

For Beginners: This is the core analysis method. It takes a long list of parameters and intelligently groups them for efficient communication.

Strategy: 1. Start with natural boundaries (we assume every N parameters belong together) 2. Merge groups that are too small 3. Align group boundaries to make distribution across processes easier

CalculateDistributionStats(List<ParameterGroup>)

Calculates statistics about parameter distribution for a model.

public Dictionary<string, double> CalculateDistributionStats(List<ParameterAnalyzer<T>.ParameterGroup> groups)

Parameters

groups List<ParameterAnalyzer<T>.ParameterGroup>

The parameter groups to analyze

Returns

Dictionary<string, double>

A dictionary of statistics (e.g., "TotalGroups", "AverageGroupSize")

Remarks

For Beginners: This gives you information about how parameters would be distributed, helping you understand the efficiency of the grouping strategy.

ValidateGrouping(List<ParameterGroup>, int)

Validates that parameter groups cover all parameters without gaps or overlaps.

public bool ValidateGrouping(List<ParameterAnalyzer<T>.ParameterGroup> groups, int totalParameterCount)

Parameters

groups List<ParameterAnalyzer<T>.ParameterGroup>

The parameter groups to validate

totalParameterCount int

The total number of parameters

Returns

bool

True if grouping is valid, false otherwise

Remarks

For Beginners: This is a safety check to make sure our grouping is correct. It verifies: - Every parameter is in exactly one group - Groups don't overlap - There are no gaps between groups

Exceptions

InvalidOperationException

Thrown if validation fails with details