Table of Contents

Class CodeSynthesisArchitecture<T>

Namespace
AiDotNet.ProgramSynthesis.Models
Assembly
AiDotNet.dll

Defines the architecture configuration for code synthesis and understanding models.

public class CodeSynthesisArchitecture<T> : NeuralNetworkArchitecture<T>

Type Parameters

T

The numeric type used for calculations (e.g., double, float).

Inheritance
CodeSynthesisArchitecture<T>
Inherited Members

Remarks

CodeSynthesisArchitecture extends the neural network architecture with code-specific parameters such as programming language, maximum code length, vocabulary size, and synthesis strategy. It serves as a blueprint for building code models like CodeBERT, GraphCodeBERT, and CodeT5.

For Beginners: This is a blueprint for building AI models that understand code.

Just like TransformerArchitecture defines how to build a general transformer, CodeSynthesisArchitecture defines how to build models specifically for:

  • Understanding code
  • Generating code
  • Translating between programming languages
  • Finding bugs
  • Completing code

It includes all the settings needed to build these specialized code models, like which programming language to work with and how much code it can handle.

Constructors

CodeSynthesisArchitecture(SynthesisType, ProgramLanguage, CodeTask, int, int, int, int, int, int, int, int, double, bool, bool, NetworkComplexity, int, int, List<ILayer<T>>?)

Initializes a new instance of the CodeSynthesisArchitecture<T> class.

public CodeSynthesisArchitecture(SynthesisType synthesisType, ProgramLanguage targetLanguage, CodeTask codeTaskType, int numEncoderLayers = 6, int numDecoderLayers = 0, int numHeads = 8, int modelDimension = 512, int feedForwardDimension = 2048, int maxSequenceLength = 512, int vocabularySize = 50000, int maxProgramLength = 100, double dropoutRate = 0.1, bool usePositionalEncoding = true, bool useDataFlow = false, NetworkComplexity complexity = NetworkComplexity.Medium, int inputSize = 0, int outputSize = 0, List<ILayer<T>>? layers = null)

Parameters

synthesisType SynthesisType

The type of synthesis approach.

targetLanguage ProgramLanguage

The target programming language.

codeTaskType CodeTask

The primary code task type.

numEncoderLayers int

Number of encoder layers.

numDecoderLayers int

Number of decoder layers.

numHeads int

Number of attention heads.

modelDimension int

Size of token embeddings.

feedForwardDimension int

Size of feed-forward layers.

maxSequenceLength int

Maximum input sequence length.

vocabularySize int

Size of the code vocabulary.

maxProgramLength int

Maximum length of synthesized programs.

dropoutRate double

Dropout rate for regularization.

usePositionalEncoding bool

Whether to use positional encoding.

useDataFlow bool

Whether to use data flow analysis.

complexity NetworkComplexity

Overall network complexity.

inputSize int

Input size (calculated from vocabulary).

outputSize int

Output size (calculated from task).

layers List<ILayer<T>>

Optional custom layers.

Remarks

Creates a new code synthesis architecture with the specified parameters. This configuration will be used to build code understanding and generation models.

For Beginners: This constructor sets up all the parameters for a code model.

When creating a code model, you specify:

  • What approach to use (neural, symbolic, etc.)
  • Which language to work with
  • What task to perform
  • How big and powerful the model should be

Many parameters have sensible defaults, so you only need to set the ones that matter for your specific use case.

Properties

CodeTaskType

Gets the code task type this architecture is optimized for.

public CodeTask CodeTaskType { get; }

Property Value

CodeTask

Remarks

Specifies the primary task this model will perform, which affects the model structure and training approach.

For Beginners: This is the main job the model will do.

Code models can do many things:

  • Complete code as you type
  • Find bugs
  • Translate between languages
  • Generate documentation

This setting optimizes the model for one specific task.

DropoutRate

Gets the dropout rate for regularization.

public double DropoutRate { get; }

Property Value

double

Remarks

The probability of dropping neurons during training to prevent overfitting.

For Beginners: This helps prevent the model from memorizing too much.

Dropout randomly disables some neurons during training, which:

  • Prevents overfitting (memorizing training data)
  • Makes the model more robust
  • Improves generalization to new code

Typical value: 0.1 (10% of neurons randomly disabled during training).

FeedForwardDimension

Gets the feed-forward network dimension.

public int FeedForwardDimension { get; }

Property Value

int

Remarks

The size of the intermediate layer in the feed-forward networks within each transformer layer. Usually 2-4 times the model dimension.

For Beginners: This is the processing power in each layer.

After attention, each layer has a feed-forward network that processes the information. This dimension controls its size:

  • Larger: More processing power
  • Smaller: Faster but less capable

Typical: 4 × ModelDimension (e.g., if ModelDim is 512, this would be 2048).

MaxProgramLength

Gets the maximum allowed program length for synthesis.

public int MaxProgramLength { get; }

Property Value

int

Remarks

Limits the size of programs that can be synthesized, measured in abstract syntax tree nodes or lines of code.

For Beginners: This limits how long generated programs can be.

Prevents the AI from creating huge, unwieldy programs. Like a word limit on an essay - keeps the output manageable and focused.

MaxSequenceLength

Gets the maximum sequence length (in tokens).

public int MaxSequenceLength { get; }

Property Value

int

Remarks

The maximum number of code tokens the model can process at once. Longer sequences capture more context but require more memory and computation.

For Beginners: This is the maximum length of code the model can handle.

Code is broken into tokens (like words). This limits how many tokens:

  • 512 tokens: ~200-400 lines of code
  • 1024 tokens: ~400-800 lines of code
  • 2048 tokens: ~800-1600 lines of code

Longer files need to be split into chunks.

ModelDimension

Gets the model dimension (embedding size).

public int ModelDimension { get; }

Property Value

int

Remarks

The size of the vector used to represent each token in the code. Larger dimensions can capture more information but require more memory.

For Beginners: This is how much information each code piece holds.

Each word/token in code is represented by a vector of numbers. This dimension controls the size of that vector:

  • Larger: Can capture more nuanced meaning
  • Smaller: Faster but less detailed

Typical values: 256-768 for code models.

NumDecoderLayers

Gets the number of decoder layers (for generation tasks).

public int NumDecoderLayers { get; }

Property Value

int

Remarks

The number of transformer decoder layers used to generate code. Only relevant for encoder-decoder models like CodeT5.

For Beginners: This controls how the model generates code.

Decoder layers are used when the model needs to create new code:

  • For code completion
  • For code translation
  • For code generation from descriptions

Not all models need decoders - some only understand code (encoders only).

NumEncoderLayers

Gets the number of encoder layers.

public int NumEncoderLayers { get; }

Property Value

int

Remarks

The number of transformer encoder layers used to process and understand code. More layers allow for deeper understanding but require more computation.

For Beginners: This controls how deeply the model analyzes code.

More encoder layers mean:

  • Better understanding of complex code patterns
  • Can capture more subtle relationships
  • Takes more time and memory to process

Typical values: 6-12 layers for code models.

NumHeads

Gets the number of attention heads.

public int NumHeads { get; }

Property Value

int

Remarks

The number of parallel attention mechanisms in each layer. More heads allow the model to focus on different aspects of code simultaneously.

For Beginners: This is how many different things the model looks at simultaneously.

Multiple attention heads let the model focus on:

  • Variable definitions
  • Function calls
  • Control flow
  • Data dependencies All at the same time!

Typical values: 8-16 heads.

SynthesisType

Gets the type of synthesis approach to use.

public SynthesisType SynthesisType { get; }

Property Value

SynthesisType

Remarks

Specifies whether to use neural, symbolic, hybrid, or genetic programming approaches for code synthesis.

For Beginners: This chooses the strategy for generating code.

Different approaches work better for different problems:

  • Neural: Good for learning from examples
  • Symbolic: Good for following rules
  • Hybrid: Combines both approaches
  • GeneticProgramming: Good for optimization problems

TargetLanguage

Gets the target programming language.

public ProgramLanguage TargetLanguage { get; }

Property Value

ProgramLanguage

Remarks

Specifies which programming language the model is designed to work with.

For Beginners: This is which programming language the model knows.

Like a translator specializing in French or Spanish, code models often specialize in specific languages like Python or Java.

UseDataFlow

Gets whether to use data flow information (for GraphCodeBERT-style models).

public bool UseDataFlow { get; }

Property Value

bool

Remarks

If true, the model will use graph-based representations that capture data flow between variables and functions, not just sequential structure.

For Beginners: This makes the model understand how data flows through code.

Beyond just reading code line by line, this tracks:

  • Which variables depend on which others
  • How data flows from one function to another
  • The relationships between different parts of code

Like understanding not just the words in a recipe, but how ingredients flow from one step to the next. Used in GraphCodeBERT models.

UsePositionalEncoding

Gets whether to use positional encoding.

public bool UsePositionalEncoding { get; }

Property Value

bool

Remarks

Determines if positional information should be added to token embeddings to help the model understand code order and structure.

For Beginners: This helps the model understand code order.

Without this, the model wouldn't know if "a = b" comes before or after "b = 5". Positional encoding adds location information so the model understands:

  • Which line comes first
  • How far apart two statements are
  • The sequential structure of code

Usually set to true for code models.

VocabularySize

Gets the vocabulary size.

public int VocabularySize { get; }

Property Value

int

Remarks

The number of unique tokens (keywords, operators, identifiers, etc.) in the model's vocabulary. Larger vocabularies can represent more code patterns.

For Beginners: This is the model's dictionary size for code.

How many different code tokens the model knows:

  • Keywords: if, for, while, class, etc.
  • Operators: +, -, ==, etc.
  • Common identifiers and patterns

Typical values: 30,000-50,000 tokens for code models.

Edit this page