Features Desired for 1.0 Release

[andrewjong]

Real Time Efficiency

• Isaac Sim is wayy slow. We need to figure out how to run things in real time
• maybe we abstract out PX4 with Isaac Sim?
• launch minimal set of autonomy/sensors/sim

Automated Testing and Reliability

make easy to observe: how does performance change under different parameters?

Random seed is fixed always so all these repeated tests if you randomize start or end are fixed either by seed or by saving the tasks to ensure consistent testing.

Triggered on every minor version change or triggered manually. Runs on airstack cloud.
Hooks to github generating report of all tests binary results as well as numerical results (50% success, 5meter deviation from goal...etc.)

Must hook up this CI/CD to AirLab Cloud

Testing Infrastructure Reliability

@OasisArtisan #16

• clone and run non-CMU from scratch
• Docker builds (Report individual docker build failures which docker image failed)
• Airstack up runs with all sims. All processes are running (Tmux windows that should have a process running have the process running for 60 seconds, otherwise report the process last 50 line output and report failure).
• ROS2 comm check. Send ros2 hello world messages across robot containers, sim container, and gcs container. May need to add a special topic to the bridge. Idk what other PX4, mavlink, or TCP comm checks we can do here.

Testing Robot Autonomy Task Performance With Benchmarks

@pvkumara

• Autonomy stack sanity check
  • (Takeoff&land) Start airstack takeoff land takeoff land takeoff land. Repeat 10 times. Ensure successfull takeoff by looking at robot location.
• performance on various tasks for task performance with metrics? e.g. how well do we Search for semantic queries? how well do we do Coverage? how well do we Navigate from point a to point b? how well do we Follow reference Trajectory? how safe is the robot behaving throughout?
  • (Trajectory following) Start airstack, takeoff, fixed trajectory, land. Repeat 10 times changing fixed trajectory (env chosen to not have obstacles for these trajectories). Possibly report time to execute path as well and deviation from path or something.
  • (Collision avoidance). Have something similar to block world in airsim. simple env. set goal beyond obstacle report success. Repeat 10 times possibly changing env or start and goal points.
  • (Frontier exploration) TBD if this should be part of the core stack but I would argue yes its basic enough.

Testing Isaac Sim Reliability

@JohnYanxinLiu

• ensure the robot.usd files are always working to drag in to scenes
• ensure the provided scenes always working
• ensure always runs doesn't crash
• ensure opening/closing things doesn't crash

Tests Integrate with GitHub CI/CD

@andrewjong

• connect with GitHub so that PRs automatically trigger these and ephemeral self hosted instances runr on AirLab Cloud

GUI overhaul

@krrishj18 @andrewjong

• we want to support multi-agent GUI on GCS that respects network bandwidth constraints
• want to underlay the satellite map on the ground plane and/or DEM plane
  • is it worth doing registration of detected features to the ground plane?
• I think supporting TAKx is out of scope for now, let DTC team do that
• headless mode w/ Foxglove?

Task Executors

GUI Interface

• interface to activate/deactivate different executors? e.g. deactivate DROAN, activate DROAN-GL. see ros2 managed lifecycle nodes
• interface to adjust executor parameters online? e.g. DROAN forward reward weight
• interface for global params Extract persistent parameters to global rosparam instead of task inputs #340

Task Executor implementations

• coverage (lawnmower) @DevRobi Lawnmower Coverage trajectory #232
• exploration (frontiers, Junbin's) @YuanJunbin Frontier Exploration Geometric Planner #341
• semantic search (RAVEN) @krrishj18 Integrate RAVEN planner for Semantic Search Task #238

Mission Sequence / Behavior Tree Task

@andrewjong

• one option may be to have a simple sequence based taks-after-task execution
• put the BTree back in with BehaviorTree.cpp, but the BTree will be implemented as a Task. The BTree Task will enable loading a custom BTree file to run, and let the user interactively construct a BTree through GUI
• what nodes do we wish to support with the BTree? every task? what about state? do we want to support activating/deactivating tasks?

Multi agent support

@krrishj18

• can task multiple agents to split up a task between them?
• Krrish's gossip protocol
• simulate communication constraints
• multi-computer to split per-robot autonomy load

Realistic/Degradation-Resistant State Estimation

• desire to integrate MACVIO/MAC-SLAM with realistic usage
• currently we're not doing super realistic state estimation, it's coming from mavros sim with simulated sensors. see px4_mavlink_backend.py. The default multirotor uses default simulated sensors including barometer, imu, magnotometer, gps. The GPS sensor for example doesn't degrade based on environment occlusions
• therefore currently the simulator won't accurately reflect performance in GPS-degraded environments
• apparently there is some research on how to simulate this: https://arc.aiaa.org/doi/full/10.2514/6.2023-2648
• need to be able to balance autonomy w/ GPS mode vs GPS-denied mode. GPS-denied important for INDOOR FLIGHT
• for GPS mode, do we want to solely rely on GPS, or do we want to fuse with MACVO/MACVIO?
• for GPS-denied mode, the truth is that it will always always eventually drift, and so there should be some correction with GPS when it becomes reliable
• this is rather complicated, we need someone dedicated to this task

Real World Deployment

@jfkeller @andrewjong

• modal AI drone, verify runs basic autonomy and obstacle avoidance
• verify onboard/offboard autonomy split
• we need an easy reliable URDF export pipeline from whatever software to airstack. maybe from isaac sim?
• need some type of consistent flash / hardware install script
• verify real world multi-agent control and communication

Documentation

• Beginner friendly tutorials for common workflows
  • setup and get started
  • run the baseline task executors / autonomy algorithms
  • run a complex mission with BehaviorTree, define a custom mission in GUI
  • create custom simulated map, e.g. from Unreal Engine, and Digital Twin
  • custom number of robots
  • custom robot and urdf export
  • custom autonomy algo
  • define new task
  • define new benchmark and metrics
  • benchmark with HITL with sim, a Jetson, and external GCS on a physical router
  • deploy to Modal AI drone and real world flight test
• Caveats with the Known Limitations of this Baseline stack, so that people don't crash this in real life. Every algorithm needs a Limitations section
• FAQ of AirStack. E.g.: is AirStack the solution to AGI? NO it's not, it's just a boilerplate and baseline. must I follow the current way AirStack is laid out? NO, customize it however you wish

[andrewjong]

Add a px4 only sim?

[andrewjong]

isaac lab multirotor has better parallelization?

[andrewjong]

task list preconditions btree?