Class ShardingConfig

Configuration for model sharding.

public class ShardingConfig

Inherited Members: object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Properties

Custom device assignments when using ShardingStrategy.Custom. Index is layer index, value is device index.

public int[]? CustomDeviceAssignments { get; set; }

Memory limit per device in bytes (for memory-balanced sharding).

public long? MaxMemoryPerDevice { get; set; }

Number of micro-batches for pipeline parallelism.

public int MicroBatchCount { get; set; }

Higher values improve GPU utilization but increase memory usage. Typical values: 4-8 for most models.

Strategy for distributing layers across devices.

public ShardingStrategy Strategy { get; set; }

Whether to use pipeline parallelism for overlapping compute and transfer.

public bool UsePipelineParallelism { get; set; }