Enum RelationModuleType
Types of relation module architectures for Relation Networks.
public enum RelationModuleTypeFields
Attention = 2Uses attention mechanism to relate features.
Uses attention to compute weighted relationships between different parts of the feature embeddings.
For Beginners: This allows the network to focus on the most important parts of each example when comparing them, rather than treating all features equally.
Concatenate = 0Concatenates features and passes through MLP.
The simplest approach: concatenate the two feature vectors and pass through a multi-layer perceptron (MLP) that outputs a relation score.
For Beginners: This is like putting two descriptions side by side and asking a neural network "how similar are these?" The network learns to look at both descriptions together and output a similarity score.
Convolution = 1Stacks features and applies 2D convolution.
Stacks the two feature maps spatially and applies 2D convolution layers to learn local patterns that indicate similarity.
For Beginners: This is useful for image data where spatial patterns matter. It's like overlaying two images and looking for patterns in how they match or differ.
Transformer = 3Uses transformer-style self-attention.
Applies transformer-style multi-head self-attention for computing relations, allowing complex feature interactions.
For Beginners: This uses the same powerful attention mechanism found in models like GPT and BERT, allowing very sophisticated comparisons between features at multiple levels.
Remarks
The relation module learns to compare feature embeddings and output a similarity score. Different architectures provide different ways of computing this learned similarity.
For Beginners: This determines HOW the network compares two examples. Instead of using a fixed formula (like Euclidean distance), Relation Networks learn a neural network to measure "how related" two examples are. This enum controls the architecture of that comparison network.