Principles and Practices of Engineering Artificially Intelligent Systems
📖 Read Online • Tiny🔥Torch • 📄 Download PDF • 📓 Download EPUB • 🌐 Explore Ecosystem
📚 Hardcopy edition coming 2026 with MIT Press.
The world is rushing to build AI systems. It is not engineering them.
AI engineering is the discipline of building efficient, reliable, safe, and robust intelligent systems that operate in the real world, not just models in isolation.
Our mission: Establish AI engineering as a foundational discipline, alongside software engineering and computer engineering, by teaching how to design, build, and evaluate end-to-end intelligent systems. The long-term impact of AI will be shaped by engineers who can turn ideas into working, dependable systems.
This repository is the open learning stack for AI systems engineering. It brings together the textbook, TinyTorch, TinyML hardware kits, and upcoming Co labs that connect system principles to running code and real devices.
The ecosystem began in Harvard’s CS249r course and has grown to support classrooms, study groups, and independent learners around the world.
We focus on turning principles into practice, moving deliberately from theory to software to hardware.
Follow the learning path that matches your goals:
READ → Start with the textbook. Try Chapter 1 and the Benchmarking chapter.
BUILD → Follow the TinyTorch getting started guide. Begin with Module 01 (Tensors) and work your way up to training CNNs and transformers with code you wrote from scratch.
DEPLOY → Pick a hardware kit and run the labs. Ship real models to Arduino, Raspberry Pi, or other edge devices.
Say hello → Introduce yourself in Discussions. We will do our best to reply.
Read the textbook, then pick your path:
┌───────────────────────────────────────────────────────────────────────────────┐
│ │
│ MACHINE LEARNING SYSTEMS │
│ Read the Textbook │
│ │
│ Theory • Concepts • Best Practices │
│ │
└───────────────────────────────────────┬───────────────────────────────────────┘
│
┌─────────────┼─────────────┐
│ │ │
▼ ▼ ▼
┌───────────────────────────────────────────────────────────────────────────────┐
│ HANDS-ON ACTIVITIES │
│ (pick one or all) │
│ │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
│ │ │ │ │ │ │ │
│ │ SOFTWARE │ │ TINYTORCH │ │ HARDWARE │ │
│ │ CO-LABS │ │ FRAMEWORK │ │ LABS │ │
│ │ │ │ │ │ │ │
│ │ EXPLORE │ │ BUILD │ │ DEPLOY │ │
│ │ │ │ │ │ │ │
│ │ Run controlled │ │ Understand │ │ Engineer under │ │
│ │ experiments on │ │ frameworks by │ │ real constraints│ │
│ │ latency, memory,│ │ implementing │ │ memory, power, │ │
│ │ energy, cost │ │ them │ │ timing, safety │ │
│ │ │ │ │ │ │ │
│ │ (coming 2026) │ │ │ │ Arduino, Pi │ │
│ └─────────────────┘ └─────────────────┘ └─────────────────┘ │
│ │
│ EXPLORE BUILD DEPLOY │
│ │
└───────────────────────────────────────┬───────────────────────────────────────┘
│
▼
┌───────────────────────────────────────────────────────────────────────────────┐
│ │
│ AI OLYMPICS │
│ Prove Mastery │
│ │
│ Compete across all tracks • University teams • Public leaderboards │
│ │
│ (coming 2026) │
│ │
└───────────────────────────────────────────────────────────────────────────────┘
| Component | What You Do | Link | |
|---|---|---|---|
| READ | 📖 Textbook | Understand ML systems concepts | book/ |
| EXPLORE | 🔮 Software Co-Labs | Run controlled experiments on latency, memory, energy, cost | Coming 2026 |
| BUILD | 🔥 TinyTorch | Understand frameworks by implementing them | tinytorch/ |
| DEPLOY | 🔧 Hardware Kits | Engineer under real constraints: memory, power, timing, safety | kits/ |
| PROVE | 🏆 AI Olympics | Compete and benchmark across all tracks | Coming 2026 |
What each path teaches:
- EXPLORE teaches why — Understand tradeoffs. Change batch sizes, precision, model architectures and see how latency, memory, and accuracy shift.
- BUILD teaches how — Understand internals. Implement autograd, optimizers, and attention from scratch to see how TensorFlow and PyTorch actually work.
- DEPLOY teaches where — Understand constraints. Face real memory limits, power budgets, and latency requirements on actual hardware.
This textbook teaches you to think at the intersection of machine learning and systems engineering. Each chapter bridges algorithmic concepts with the infrastructure that makes them work in practice.
| ML Concept | Systems Concept | What You Learn |
|---|---|---|
| Model parameters | Memory constraints | How to fit large models on resource-limited devices |
| Inference latency | Hardware acceleration | How GPUs, TPUs, and accelerators execute neural networks |
| Training convergence | Compute efficiency | How mixed-precision and optimization techniques reduce cost |
| Model accuracy | Quantization and pruning | How to compress models while preserving performance |
| Data requirements | Pipeline infrastructure | How to build efficient data loading and preprocessing |
| Model deployment | MLOps practices | How to monitor, version, and update models in production |
| Privacy constraints | On-device learning | How to train and adapt models without sending data to the cloud |
| Part | Focus | Chapters |
|---|---|---|
| I. Foundations | Core concepts | Introduction, ML Systems, DL Primer, Architectures |
| II. Design | Building blocks | Workflow, Data Engineering, Frameworks, Training |
| III. Performance | Making it fast | Efficient AI, Optimizations, HW Acceleration, Benchmarking |
| IV. Deployment | Making it work | MLOps, On-device Learning, Privacy, Robustness |
| V. Trust | Making it right | Responsible AI, Sustainable AI, AI for Good |
| VI. Frontiers | What's next | Emerging trends and future directions |
This is a living textbook. We keep it updated as the field grows, with community input along the way.
AI may feel like it is moving at lightning speed, but the engineering building blocks that make it work do not change as quickly as the headlines. This project is built around those stable foundations.
Think of it like LEGO. New sets arrive all the time, but the bricks themselves stay the same. Once you learn how the bricks fit together, you can build anything. Here, those "AI bricks" are the solid systems principles that make AI work.
Whether you are reading a chapter, running a lab, or sharing feedback, you are helping make these ideas more accessible to the next learner.
We use the same loop for research and teaching: define the system problem, build a reference implementation, benchmark it, then turn it into curriculum and tooling so others can reproduce and extend it.
| Loop Step | Research Artifacts | Teaching Artifacts |
|---|---|---|
| Measure | Benchmarks, suites, metrics | Benchmarking chapter, assignments |
| Build | Reference systems, compilers, runtimes | TinyTorch modules, co-labs |
| Deploy | Hardware targets, constraints, reliability | Hardware labs, kits |
We are tracking our way to 1 million learners by 2030. Every star, share, and contribution helps us get there.
What gets measured gets done. Every star represents a learner or believer in accessible AI education.
1 ✓ → 10 ✓ → 100 ✓ → 1,000 ✓ → 10,000 ✓ → 100,000 by 2026 → ... → 1M by 2030
Here's how it works: Organizations like the EDGE AI Foundation watch open source education projects. When they see a growing, active community, they contribute funding. Your star is not just a bookmark. It is a vote that sponsors notice.
That funding goes directly to Open Collective and supports initiatives like TinyML4D workshops in developing countries, TinyML kits for underserved classrooms, and infrastructure to keep this resource free for everyone.
One click. Real impact.
All contributions go to Open Collective, a transparent fund that supports educational outreach.
If you want to help grow AI systems engineering as a discipline through curriculum, benchmarks, and hands-on tooling, reach out via Discussions or Open Collective.
| Resource | Description |
|---|---|
| 📖 Textbook | Interactive online textbook |
| 🔥 TinyTorch | Build ML frameworks from scratch |
| 🔧 Hardware Kits | Deploy to Arduino, Raspberry Pi, edge devices |
| 🌐 Ecosystem | Resources, workshops, and community |
| 💬 Discussions | Questions and ideas |
We welcome contributions to the book, TinyTorch, and hardware kits!
| I want to... | Go here |
|---|---|
| Fix a typo or improve a chapter | book/docs/CONTRIBUTING.md |
| Add a TinyTorch module or fix a bug | tinytorch/CONTRIBUTING.md |
| Improve hardware labs | kits/README.md |
| Report an issue | GitHub Issues |
| Ask a question | GitHub Discussions |
@inproceedings{reddi2024mlsysbook,
title = {MLSysBook.AI: Principles and Practices of Machine Learning Systems Engineering},
author = {Reddi, Vijay Janapa},
booktitle = {2024 International Conference on Hardware/Software Codesign and System Synthesis (CODES+ ISSS)},
pages = {41--42},
year = {2024},
organization = {IEEE},
url = {https://mlsysbook.org}
}This project uses a dual-license structure:
| Component | License | What It Means |
|---|---|---|
| Book content | CC BY-NC-ND 4.0 | Share freely with attribution; no commercial use; no derivatives |
| TinyTorch code | Apache 2.0 | Use, modify, and distribute freely; includes patent protection |
The textbook content (chapters, figures, explanations) is educational material that should circulate with attribution and without commercial exploitation. The software framework is a tool designed to be easy for anyone to use, modify, or integrate into their own projects.
Thanks goes to these wonderful people who have contributed to making this resource better for everyone:
⭐ Star us on GitHub • ✉️ Subscribe • 💬 Join discussions • 🌐 Visit mlsysbook.ai
Built with dedication by the MLSysBook community.