micrograd-go

A tiny, Go-native autograd engine that implements backpropagation over a dynamically built DAG. It is a Go port of Andrej Karpathy's micrograd.

It features a scalar automatic differentiation engine (Value) and a small neural network library on top of it (Neuron, Layer, MLP).

Installation

go get github.com/tridipdam11/micrograd-go

Quick Start

Using the Autograd Engine

You can build computational graphs using the Value struct and compute gradients with backpropagation.

package main

import (
	"fmt"

	"github.com/tridipdam11/micrograd-go/grad"
)

func main() {
	a := grad.NewValue(2.0)
	b := grad.NewValue(-3.0)
	c := grad.NewValue(10.0)
	
	// e = a * b
	e := a.Mul(b)
	// d = e + c
	d := e.Add(c)
	// f = d.ReLU()
	f := d.ReLU()
	
	f.Backward()
	
	fmt.Println("f = ", f.Data)    // Output: 4.0
	fmt.Println("a.Grad = ", a.Grad) // Output: -3.0
	fmt.Println("b.Grad = ", b.Grad) // Output: 2.0
}

Building a Neural Network

micrograd-go provides basic building blocks for constructing Multi-Layer Perceptrons (MLPs).

package main

import (
	"fmt"

	"github.com/tridipdam11/micrograd-go/grad"
)

func main() {
	// Create an MLP with 3 inputs, two hidden layers of size 4, and 1 output
	mlp := grad.NewMLP(3, []int{4, 4, 1})

	// Define training data
	xs := [][]*grad.Value{
		{grad.NewValue(2.0), grad.NewValue(3.0), grad.NewValue(-1.0)},
		{grad.NewValue(3.0), grad.NewValue(-1.0), grad.NewValue(0.5)},
		{grad.NewValue(0.5), grad.NewValue(1.0), grad.NewValue(1.0)},
		{grad.NewValue(1.0), grad.NewValue(1.0), grad.NewValue(-1.0)},
	}
	ys := []*grad.Value{
		grad.NewValue(1.0),
		grad.NewValue(-1.0),
		grad.NewValue(-1.0),
		grad.NewValue(1.0),
	}

	// Simple gradient descent loop
	for k := 0; k < 20; k++ {
		// Forward pass
		var ypred []*grad.Value
		for _, x := range xs {
			ypred = append(ypred, mlp.Forward(x)[0])
		}

		// Compute loss (mean squared error)
		loss := grad.NewValue(0.0)
		for i := 0; i < len(ys); i++ {
			diff := ypred[i].Add(ys[i].Mul(grad.NewValue(-1.0)))
			loss = loss.Add(diff.Pow(2.0))
		}

		// Zero gradients
		for _, p := range mlp.Parameters() {
			p.Grad = 0.0
		}

		// Backward pass
		loss.Backward()

		// Update parameters (gradient descent)
		learningRate := -0.05
		for _, p := range mlp.Parameters() {
			p.Data += learningRate * p.Grad
		}

		fmt.Printf("Step %d: loss = %f\n", k, loss.Data)
	}
}

Features

• Dynamic Graph Construction: Graph builds implicitly as you perform operations on Value objects.
• Fluent Method Chaining: Methods like .Add(), .Mul(), .Pow(), and .ReLU() allow writing mathematical expressions naturally in Go.
• Reverse-Mode Backpropagation: Calls .Backward() to compute partial derivatives recursively.
• Standard Activation Functions: Supports ReLU activation out-of-the-box.

License

MIT