Add HaloBlocks: composable PyTorch model component library

[Copilot]

Bootstrap the haloblocks Python library — every model component is a composable Block (subclass of nn.Module) that can be freely mixed to build custom architectures.

Core blocks

• Base — Block: common interface + num_parameters(trainable_only=False)
• Layers — LinearBlock, LayerNormBlock, DropoutBlock, FeedForwardBlock (GELU/ReLU), EmbeddingBlock
• Attention — MultiHeadAttentionBlock, SelfAttentionBlock (optional causal mask), CrossAttentionBlock
• Encoders — TransformerEncoderLayerBlock, TransformerEncoderBlock (stacked)
• Decoders — TransformerDecoderLayerBlock (masked self-attn + cross-attn), TransformerDecoderBlock (stacked)
• Heads — ClassificationHead (cls/mean pooling), LanguageModelHead (weight tying), TokenClassificationHead

Example — full Transformer from blocks

import haloblocks as hb

class Seq2Seq(nn.Module):
    def __init__(self, vocab_size):
        super().__init__()
        self.src_emb = hb.EmbeddingBlock(vocab_size, 256)
        self.tgt_emb = hb.EmbeddingBlock(vocab_size, 256)
        self.encoder = hb.TransformerEncoderBlock(256, num_heads=8, d_ff=1024, num_layers=6)
        self.decoder = hb.TransformerDecoderBlock(256, num_heads=8, d_ff=1024, num_layers=6)
        self.head    = hb.LanguageModelHead(256, vocab_size)

    def forward(self, src, tgt):
        memory  = self.encoder(self.src_emb(src))
        decoded = self.decoder(self.tgt_emb(tgt), memory)
        return self.head(decoded)

Packaging

• pyproject.toml + requirements.txt (torch>=1.13)
• 35 unit tests across all block categories

Original prompt

python library where all model components are treated as blocks like heads,attention layers, new layers, encoders, decoders

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.