Continuous Thought Machines – Lab (CTM-Lab)

CTM-Lab is an independent, experimental research codebase inspired by the Continuous Thought Machines paper (Sakana AI). It focuses on simplified architectures, instrumentation, and task-dependent synchronization analysis.

Hybrid layout:

• src/ctm_lab/ : importable library (models, data, viz utilities)
• experiments/ : runnable entrypoints per experiment (config, train, compute_sync, plot_sync)

Install

python -m venv .venv
source .venv/bin/activate
pip install -U pip
pip install -e .

Run MNIST multitask

python experiments/mnist_multitask/train.py --config experiments/mnist_multitask/config.yaml

# Use the printed checkpoint path, or find:
# outputs/mnist_multitask/config/<run_id>/checkpoints/latest.pt

python experiments/mnist_multitask/compute_sync.py \
  --config experiments/mnist_multitask/config.yaml \
  --ckpt outputs/mnist_multitask/config/<run_id>/checkpoints/latest.pt \
  --out outputs/mnist_sync_stats.pt

python experiments/mnist_multitask/plot_sync.py \
  --config experiments/mnist_multitask/config.yaml \
  --sync outputs/mnist_sync_stats.pt \
  --outdir outputs/mnist_figs

Notes:

• Computes full D×D synchronization for small D (good for viz).
• Designed to be upgraded later (sampled sync, cross-attn, per-neuron params).

Memory frontier (Plan A)

Smoke run (tiny steps):

python -m experiments.memory_frontier.train --config experiments/memory_frontier/config.yaml \
  train.steps=2 train.val_steps=1 model.D=32 model.T=2 train.run_name=smoke_ctm

Acknowledgements and Citation

CTM-Lab is inspired by the Continuous Thought Machines work by Sakana AI. If you use ideas related to Continuous Thought Machines, please cite the original paper:

@techreport{darlow2025ctm,
  title     = {Continuous Thought Machines},
  author    = {Luke Darlow and Ciaran Regan and Sebastian Risi and Jeffrey Seely and Llion Jones},
  institution = {Sakana AI},
  year      = {2025},
  note      = {arXiv:2505.05522}
}

CTM-Lab is an independent, experimental research codebase and is not affiliated with or endorsed by Sakana AI.

If you use CTM-Lab code, experiments, or visualizations in your work, please consider citing this repository:

@software{kwon_ctm_lab_2025,
  title  = {Continuous Thought Machines - Lab (CTM-Lab)},
  author = {Kwon, Youngwook Paul},
  year   = {2025},
  url    = {https://github.com/ywpkwon/continuous-thought-machines-lab}
}