Skip to content

aniekannn/alpamayo

Β 
Β 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

7 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

πŸ”οΈ Alpamayo 1

Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving

HuggingFace arXiv License

Note: Following the release of NVIDIA Alpamayo at CES 2026, Alpamayo-R1 has been renamed to Alpamayo 1.

πŸ“– Please read the HuggingFace Model Card first! The model card contains comprehensive details on model architecture, inputs/outputs, licensing, and tested hardware configurations. This GitHub README focuses on setup, usage, and frequently asked questions.

Requirements

Requirement Specification
Python 3.12.x (see pyproject.toml)
GPU NVIDIA GPU with β‰₯24 GB VRAM (e.g., RTX 3090, RTX 4090, A5000, H100)
OS Linux (tested); other platforms unverified

⚠️ Note: GPUs with less than 24 GB VRAM will likely encounter CUDA out-of-memory errors.

Installation

1. Install uv (if not already installed)

curl -LsSf https://astral.sh/uv/install.sh | sh
export PATH="$HOME/.local/bin:$PATH"

2. Set up the environment

uv venv ar1_venv
source ar1_venv/bin/activate
uv sync --active

3. Authenticate with HuggingFace

The model requires access to gated resources. Request access here:

Then authenticate using the HuggingFace CLI:

# Install huggingface-cli if not already installed (included in transformers)
pip install huggingface_hub

# Login with your token
huggingface-cli login

Get your access token at: https://huggingface.co/settings/tokens

πŸ’‘ Tip: For more details on HuggingFace authentication, see the official documentation.

Running Inference

Test script

NOTE: This script will download both some example data (relatively small) and the model weights (22 GB). The latter can be particularly slow depending on network bandwidth. For reference, it takes around 2.5 minutes on a 100 MB/s wired connection.

python src/alpamayo_r1/test_inference.py

In case you would like to obtain more trajectories and reasoning traces, please feel free to change the num_traj_samples=1 argument to a higher number (Line 60).

Interactive notebook

We provide a notebook with similar inference code at notebook/inference.ipynb.

Relationship with the Paper

Alpamayo 1 implements the architecture described in our paper "Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail ", including:

Feature Paper Description This Release (v1.0)
Chain-of-Causation (CoC) reasoning Hybrid auto-labeling with human in the loop for reasoning traces βœ… Included
Vision-Language-Action architecture Cosmos-Reason backbone + action expert βœ… Included
Trajectory prediction 6.4s horizon, 64 waypoints at 10 Hz βœ… Included
RL post-training Reinforcement learning for reasoning/action consistency ❌ Not in this release
Route/navigation conditioning Explicit navigation or route inputs ❌ Not in this release
Meta-actions/General VQA High-level behavior and visual question answering ❌ Not in this release

The current release focuses on the core supervised learning components. RL post-training and route conditioning are potential candidates for future releases. Stay tuned!

Frequently Asked Questions (FAQ)

Does the 10B model accept navigation/route inputs?

While we have experimented with route conditioning capabilities, the released model does not include this feature. The current release takes multi-camera video and egomotion history as inputs, without explicit navigation or route inputs (e.g., waypoints, turn-by-turn navigation instructions).

Does the model produce meta-actions or support general VQA?

While we have experimented with meta-action and general VQA capabilities, the released model does not include these features. Alpamayo 1 is designed specifically for trajectory prediction with Chain-of-Causation reasoning, producing trajectory + reasoning trace outputs.

Was the 10B model post-trained with Reinforcement Learning (RL)?

No. The current 10B model release has not undergone RL post-training. While the paper describes RL stages for improving reasoning quality and action consistency, this release focuses on the supervised learning components. As mentioned above, we may release RL post-trained models in future releases.

What are the minimum GPU requirements?

You need an NVIDIA GPU with at least 24 GB VRAM for inference. Tested configurations include RTX 3090, A100, and H100. Running on GPUs with less memory (e.g., 16 GB) will likely result in CUDA out-of-memory errors.

Can I use this model in production / commercial applications?

No. The model weights are released under a non-commercial license. This release is intended for research, experimentation, and evaluation purposes only. See the License section and the HuggingFace Model Card for details.

Project Structure

alpamayo/
β”œβ”€β”€ notebook/
β”‚   └── inference.ipynb                  # Example notebook
β”œβ”€β”€ src/
β”‚   └── alpamayo_r1/
β”‚       β”œβ”€β”€ action_space/
β”‚       β”‚   └── ...                      # Action space definitions
β”‚       β”œβ”€β”€ diffusion/
β”‚       β”‚   └── ...                      # Diffusion model components
β”‚       β”œβ”€β”€ geometry/
β”‚       β”‚   └── ...                      # Geometry utilities and modules
β”‚       β”œβ”€β”€ models/
β”‚       β”‚   β”œβ”€β”€ ...                      # Model components and utils functions
β”‚       β”œβ”€β”€ __init__.py                  # Package marker
β”‚       β”œβ”€β”€ config.py                    # Model and experiment configuration
β”‚       β”œβ”€β”€ helper.py                    # Utility functions
β”‚       β”œβ”€β”€ load_physical_aiavdataset.py # Dataset loader
β”‚       β”œβ”€β”€ test_inference.py            # Inference test script
β”œβ”€β”€ pyproject.toml                       # Project dependencies
└── uv.lock                              # Locked dependency versions

Troubleshooting

Flash Attention issues

The model uses Flash Attention 2 by default. If you encounter compatibility issues:

# Use PyTorch's scaled dot-product attention instead
config.attn_implementation = "sdpa"

CUDA out-of-memory errors

If you encounter OOM errors:

  1. Ensure you have a GPU with at least 24 GB VRAM
  2. Reduce num_traj_samples if generating multiple trajectories
  3. Close other GPU-intensive applications

License

Disclaimer

Alpamayo 1 is a pre-trained reasoning model designed to accelerate research and development in the autonomous vehicle (AV) domain. It is intended to serve as a foundation for a range of AV-related use cases-from instantiating an end-to-end backbone for autonomous driving to enabling reasoning-based auto-labeling tools. In short, it should be viewed as a building block for developing customized AV applications.

Important notes:

  • Alpamayo 1 is provided solely for research, experimentation, and evaluation purposes.
  • Alpamayo 1 is not a fully fledged driving stack. Among other limitations, it lacks access to critical real-world sensor inputs, does not incorporate required diverse and redundant safety mechanisms, and has not undergone automotive-grade validation for deployment.

By using this model, you acknowledge that it is a research tool intended to support scientific inquiry, benchmarking, and explorationβ€”not a substitute for a certified AV stack. The developers and contributors disclaim any responsibility or liability for the use of the model or its outputs.

Citation

If you use Alpamayo 1 in your research, please cite:

@article{nvidia2025alpamayo,
      title={{Alpamayo-R1}: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail},
      author={NVIDIA and Yan Wang and Wenjie Luo and Junjie Bai and Yulong Cao and Tong Che and Ke Chen and Yuxiao Chen and Jenna Diamond and Yifan Ding and Wenhao Ding and Liang Feng and Greg Heinrich and Jack Huang and Peter Karkus and Boyi Li and Pinyi Li and Tsung-Yi Lin and Dongran Liu and Ming-Yu Liu and Langechuan Liu and Zhijian Liu and Jason Lu and Yunxiang Mao and Pavlo Molchanov and Lindsey Pavao and Zhenghao Peng and Mike Ranzinger and Ed Schmerling and Shida Shen and Yunfei Shi and Sarah Tariq and Ran Tian and Tilman Wekel and Xinshuo Weng and Tianjun Xiao and Eric Yang and Xiaodong Yang and Yurong You and Xiaohui Zeng and Wenyuan Zhang and Boris Ivanovic and Marco Pavone},
      year={2025},
      journal={arXiv preprint arXiv:2511.00088},
}

About

No description, website, or topics provided.

Resources

License

Contributing

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages

  • Python 95.9%
  • Jupyter Notebook 4.1%