ml

Machine Learning Examples

Design name	Data type	Description
Eltwise (Add / Mul)	bfloat16	Element-wise addition or multiplication of two vectors (op={add,mul} option).
Eltwise Unary (ReLU / SiLU / GELU)	bfloat16	Element-wise ReLU, SiLU, or GELU activation on a vector (op={relu,silu,gelu} option).
SwiGLU	bfloat16	Swish-Gated Linear Unit (SwiGLU) activation function on a vector
Softmax	bfloat16	Softmax operation on a matrix
Norm (RMS / Layer)	bfloat16	Row-wise RMSNorm or LayerNorm on a matrix (op={rms,layer} option).
RoPE	bfloat16	Rotary Position Embedding on a matrix
Scale Shift	bfloat16	Element-wise scale (multiply) and shift (add) on vectors
Conv2D (optional fused ReLU)	i8	1x1 Conv2D for CNNs; fuse_relu=1 swaps the output to uint8 saturation, fusing ReLU at the vector register level.
Conv2D 14x14	i8	A multi-core 2D convolution for 14x14 feature maps
Bottleneck	ui8	A Bottleneck Residual Block is a variant of the residual block that utilizes three convolutions, using 1x1, 3x3, and 1x1 filter sizes, respectively. The implementation features fusing of multiple kernels and dataflow optimizations, highlighting the unique architectural capabilities of AI Engines
ResNet	ui8	ResNet with offloaded conv2_x layers. The implementation features depth-first implementation of multiple bottleneck blocks across multiple NPU columns.
Magika	bfloat16	Magika file-type detection model inference on the NPU.
Block Datatypes	various	Examples demonstrating block floating point and other block datatypes on the NPU.