A benchmark is only meaningful if every method competes under the same information. This document states exactly what each part of the stack can observe, so results can't be quietly gamed. (Short answer: no controller reads true obstacle positions; reactive avoidance is lidar-only.)
| Layer | Input it may use | Notes |
|---|---|---|
| Global planner (A*) | The static occupancy grid (obs.occupancy) |
This map is currently ground-truth (no SLAM yet — a stated M0 simplification, see below). It contains only static obstacles. It does not contain moving obstacles. |
| Local controllers (pure-pursuit, DWA, MPPI, NMPC, RL) | The A* path + the lidar scan (obs.lidar) |
They perceive obstacles only through lidar. obstacle_points() derives points from beam returns; NMPC/MPPI/DWA use those; RL uses lidar "closeness". None reads true obstacle coordinates. |
| Lidar | Ray-cast against static grid and moving obstacles | Moving obstacles shorten the corresponding beams, so they are sensed, never known a priori. |
| Environment (judge) | True geometry of robot + all obstacles | Used only to score collisions/success — this is the referee, not an agent input. |
| SPL reference | Independent A* on the same static map | The "shortest path" denominator is computed identically for every stack, so SPL is comparable. |
- Identical scenarios. A fixed seed builds the same world (obstacles, start, goal, moving-obstacle trajectories) for every stack. Controllers are compared on the same episodes, not luck.
- Same global plan. Every stack gets the same A* path from the same known static map; the comparison isolates the local controller.
- Moving obstacles are not pre-known. The planner cannot route around movers (you can't pre-plan an
unpredictable agent); only the lidar-driven controller can react. This is exactly why the
dynamicscenario separates reactive controllers (MPPI/DWA/RL) from pure path-tracking (pure-pursuit).
By default the global planner uses a ground-truth static map (the occupancy grid is given, not built from sensors). This is standard in navigation research, and it is identical for all stacks, so it does not bias the classical-vs-learned comparison.
It is no longer required: the slam / mapping stacks build the occupancy map online from lidar
(and, with localize: true, estimate their own pose by odometry + correlative scan matching — never
reading the true pose after the start). The realism cost is then a measured quantity (EXP-0013):
built-map navigation is nearly free (the robot discovers obstacles and replans), while full SLAM's
localization drift costs real success — a feature, reported, not a cheat.
- ❌ No controller is special-cased in the runner, env, or metrics (interface-only).
- ❌ No controller is fed true obstacle positions, velocities, or the goal-optimal action.
- ❌ No per-seed tuning; no training on evaluation seeds (RL trains on a disjoint randomized distribution; eval densities are in-support but the exact seeds are never trained on).
- ❌ No silent truncation of sweeps/results (logged if it ever happens — architectural rule #5).