Skip to content

Home

Jump to bottom
skaiui2 edited this page Apr 4, 2026 · 1 revision

Welcome to the lttit wiki!

Introduction

Introduction

LTTIT is a dynamically programmable distributed operating system.

My goal is simple but ambitious:

to make one hundred microcontrollers behave like a single machine.

To achieve this, I built LTTit.

This project is not an experiment or a toy.
It is an exploration into what a distributed operating system could be.

Project Structure

This section gives a quick overview of the repository layout to help readers navigate the source code or port individual components.

LTTIT can be decomposed into more than a dozen modules.
In the GitHub repository, the demo directory contains a dual‑node example for version 0.1, while lttit is the main project directory.

The structure is as follows:

├─ccBPF  
│  ├─compiler        C-subset compiler
│  │  ├─backend      Backend
│  │  ├─frontend     Frontend
│  │  └─ir           Intermediate representation
│  └─vm              
│      └─bpf         BPF virtual machine
├─cluster            Global node tree
├─CSC                Distributed communication stack
│  ├─ccnet           Routing protocol
│  ├─ccrpc           Remote procedure call protocol
│  └─scp             Reliable transport protocol
├─fs                 File system
├─lib                Data structures and math utilities
├─mg                 Memory management
├─RTOS               Real-time microkernel
├─shell              Interactive shell
├─TcpIp              TCP/IP protocol stack
├─vfs                Virtual file system
└─vim                A lightweight text editor

Design Philosophy

Abstraction of Space and Time

The distinction between “local” and “remote” is fundamentally artificial.
Sending a message over a network versus accessing a device over a hardware bus differs only in:

  1. Distance (local is near, remote is far)
  2. Latency (local is fast, remote is slow)

But what if remote access becomes almost as fast as local access?

Then:

  • Remote memory can be treated as local memory
  • Instructions can be fetched from remote nodes
  • A cluster of microcontrollers can behave like a multi-core processor

Thus, the difference between local and remote is quantitative, not qualitative.

Transmission

UART, SPI, I²C, Ethernet—under our abstraction, these are all the same:

Data moves from P1 → P2, and it takes some amount of time.

Therefore, a communication protocol only needs to guarantee:

  1. Deliver data intact
  2. Deliver it as fast as possible

Radical Distribution

LTTIT adopts a far more radical distributed model than traditional operating systems.

Most OSes treat distribution as a network extension.
LTTIT treats distribution as a core architectural principle.

Because components are modular, the entire OS does not need to run on a single device.
For example:

  1. Node A runs the file system
  2. Node B runs TCP/IP and network drivers

When Node A wants to access the Internet:

  • It sends a message to Node B via the distributed bus
  • Node B forwards the packet to the external network
  • The response is routed back to Node A

Similarly, Node B can write persistent data directly into Node A’s file system.

        +------------------+
        |     Leader       |
        |  (World Tree)    |
        +--------+---------+
                 |
     --------------------------------
     |                              |
+----+-----+                  +------+-----+
|  Node A  |                  |  Node B    |
|  FS      |                  |  TCP/IP    |
|  BPF VM  |                  |  Drivers   |
+----------+                  +------------+

Each node has different peripherals and drivers.
All nodes register their resources to a designated leader, which maintains the global world tree.

This level of distribution would be disastrous for a general-purpose OS,
but in embedded systems with stable internal networks, it becomes practical and powerful.

Dynamic Modification

In traditional MCU development, changing logic requires:

  • Recompiling
  • Reflashing firmware
  • Or switching configurations

LTTIT eliminates this friction.

With a BPF-like compiler and virtual machine, LTTIT supports:

  • Hook points anywhere logic may change
  • Safe verification of injected programs
  • Dynamic loading of C functions as bytecode

You can:

  • Compile on Node A
  • Send the bytecode to Node B
  • Node B loads it into the appropriate hook

Examples:

  • Dynamic firewall rules
  • Traffic statistics
  • Rate limiting
  • Runtime behavior modification

No reboot, no reflashing.

Everything Is a File

To the user, each node appears as a directory.
All local and remote resources are exposed as files.

Examples:

  • /nodeA/led
  • /nodeB/sensor/temp

Each node registers its own devices.

This abstraction mirrors Plan 9, which also treats the entire system as a file hierarchy.

Microkernel

LTTIT naturally adopts a microkernel architecture.

A microkernel is not “simple”—
it is minimal, leaving space for extensibility.

All Interactions Are Message Passing

Every component and module communicates via messages.

  • A module receives input
  • Produces output
  • And the entire system becomes a network of message flows

Programs are compositions of interacting modules.
The distributed bus is the fabric that connects them.

Realizing the Philosophy

Distributed System Properties

Thanks to the microkernel and distributed communication stack, LTTIT aims to provide:

  • Single system image
    The cluster appears as one machine with unified resources.
  • Transparency
    Users do not need to know where resources reside:
    • Location transparency
    • Access transparency
    • Migration transparency
    • Expansion transparency
  • Resource sharing
    CPU, memory, devices, files, and network resources are shared.
  • Communication and coordination
    Nodes collaborate via messages, RPC, and events.
  • Independent nodes
    Each node has its own CPU, memory, and drivers.

All resources are organized into a global prefix tree:

├─cluster   Global node tree

Leader and Members

A designated leader maintains the world tree.

When a member joins:

  • It sends a registration request
  • Its resources are mounted into the world tree
  • All resources follow a unified file interface

Example:

struct vfs_ops {
    int (*open)(void *self, const char *path, int flags);
    int (*read)(void *self, int fd, void *buf, int len);
    int (*write)(void *self, int fd, const void *buf, int len);
    int (*ctl)(void *self, int fd, int cmd, void *arg);
    int (*close)(void *self, int fd);
};

Dynamic Programming and Safety

Programming in LTTIT is done through:

  • A C-subset compiler
  • A BPF virtual machine

The compiler produces bytecode, and the VM executes it.

Safety is the top priority for runtime-injected logic.

Thus:

  • The compiler is intentionally restricted
  • The VM enforces strict instruction rules
  • All programs are verified before execution

Microkernel Design

The kernel is an RTOS, but the entire LTTIT system is soft real-time.

User processes can act as:

  • Applications
  • Or OS-level services

The microkernel provides:

  • Threads
  • Clock and timers
  • Memory management
  • Interrupt handling
  • Semaphores, mutexes, and other IPC primitives

Communication Stack (CSC)

CSC is the distributed bus and RPC layer:

├─CSC
│  ├─ccnet   Routing protocol
│  ├─ccrpc   Remote procedure call
│  └─scp     Reliable transport

ccnet

A simple Dijkstra-based routing protocol ensuring delivery and integrity.

scp

A reliable transport protocol providing:

  • Congestion control
  • Flow control
  • High throughput
  • Protection against overload

ccrpc

A file-interface-based RPC mechanism:

  • open
  • read
  • write
  • close

This is the core of the “everything is a file” model.

Clone this wiki locally