AGENTS-LLM: Augmentative GENeration of Challenging Traffic Scenarios with an Agentic LLM Framework
Yu Yao5†* , Salil Bhatnagar2†*, Markus Mazzola1, Vasileios Belagiannis2, Igor Gilitschenski3,6, Luigi Palmieri1, Simon Razniewski4, and Marcel Hallgarten1
1Robert Bosch GmbH, 2Friedrich-Alexander-Universität Erlangen-Nürnberg
3University of Toronto 4ScaDS.AI & TU Dresden, 5Motional 6Vector Institute*denotes equal contribution, †work done while with Robert Bosch GmbH
International Conference on Intelligent Robots and Systems (IROS), 2025
AGENTS-LLM introduces a novel framework for augmenting real-world traffic scenarios with safety-critical edge cases using agentic large language models (LLMs).
It enables scalable, controllable, and cost-efficient scenario generation while preserving the realism of recorded driving data.
-
Augmentative Scenario Generation with LLM Agents
Rather than synthesizing scenes from scratch, AGENTS-LLM modifies existing real traffic scenarios through natural language instructions. This preserves realism while injecting rare and challenging corner cases. -
Agentic LLM Architecture for Fine-Grained Control
A structured, agent-based LLM workflow decomposes reasoning and editing steps, enabling precise, interpretable, and controllable scenario modifications — even with cost-efficient foundation models. -
Real-Data Anchored Augmentation
All generated scenarios are grounded in recorded driving data, minimizing distribution shift and maintaining physical plausibility. -
Low-Cost, High-Quality Outputs
The agentic design achieves high-quality augmentations without requiring large or expensive LLM deployments, enabling scalable usage. -
Human Expert Validation
Extensive expert evaluations demonstrate strong alignment between generated augmentations and intended modifications, matching the quality of manual expert-designed scenarios.
-
Scalable Edge-Case Generation
Dramatically reduces manual effort required to create rare, safety-critical traffic scenarios. -
Improved Planner Stress Testing
Enables more effective evaluation of autonomous driving planners by introducing controlled, challenging variations of real-world scenes. -
Bridging Real and Synthetic Data
Combines the realism of recorded datasets with the flexibility of generative methods — without fully synthetic artifacts. -
Natural Language as an Interface
Makes scenario editing accessible through intuitive text instructions, lowering the barrier for scenario design and experimentation.
[2026/02/25]initial release- scenario modification
- visualization scripts
This software is a research prototype, solely developed for and published as part of the Agents-LLM publication. It will neither be maintained nor monitored in any way.
The agents-llm package requires nuplan-devkit and interPlan.
After cloning this repository, download the nuplan-devkit and interPlan repositories and place them in the parent folder of this repository.
cd nuplan-devkit
conda env create -f environment.yml
pip install -e .
cd ../interplan-plugin
pip install -e .
cd ../agents_llm
pip install -r requirements.txt
pip install -e .
Open the file .env in the agents_llm directory and adapt the environment variables to your own setup.
Problem: Installation of nuplan-devkit is stuck at Installing pip dependencies: ...working...
Solution: update the pip version in the environment.yml file in the nuplan-devkit folder to pip=23.3.2
AGENTS-LLM is open-sourced under the AGPL-3.0 license. See the LICENSE file for details.
@inproceedings{yao2025agents,
author={Yao, Yu and Bhatnagar, Salil and Mazzola, Markus and Belagiannis, Vasileios and Gilitschenski, Igor and Palmieri, Luigi and Razniewski, Simon and Hallgarten, Marcel},
booktitle={IROS},
title={{AGENTS-LLM: Augmentative GENeration of Challenging Traffic Scenarios with an Agentic LLM Framework}},
year={2025}
}