ARCHITECTURE.md

Development guide

The goal ArduPilot methodic Configurator software is to automate some of the tasks involved in configuring and tuning ArduPilot-based vehicles. To develop the software the standard V-Model software development method was first used. It was augmented with DevOps and CI/CD automation practices at a later stage.

V-model

Development started by using the classic V-model development methodology.

flowchart TB
    subgraph requirements["Requirements & Design"]
        direction LR
        Z([User Requirements]) --> A
        A([System Requirements]) --> B([Software Architecture])
        B --> C([Module Design])
    end

    subgraph testing["Testing"]
        direction LR
        F --> G([Acceptance Testing])
        E --> F([System Testing])
        D([Module Testing]) --> E([Integration Testing])
    end

    %% V-shape connections (crossing levels)
    Z --> G
    A --> F
    B --> E
    C --> D

    %% Clickable links
    click A "SYSTEM_REQUIREMENTS.md"
    click B "#the-software-architecture"
    click C "#module-design"
    click D "#module-testing"
    click E "#integration-testing"
    click F "#system-testing"
    click G "#acceptance-testing"

    %% Styling
    classDef requirement fill:#e1f5fe,stroke:#01579b,stroke-width:2px
    classDef design fill:#f3e5f5,stroke:#4a148c,stroke-width:2px
    classDef testing fill:#e8f5e8,stroke:#1b5e20,stroke-width:2px

    class A,Z requirement
    class B,C design
    class D,E,F,G testing

Loading

If the diagram above does not display correctly, see here

The Software architecture

We decided to use python as programming language, and the following libraries and frameworks:

• pymavlink for the flight controller communication
• tkinter for the graphical user interface
• GNU gettext po files to translate the software to other languages

The (main) application itself does the command line parsing and starts the sub-applications defined below in sequence

• __main__.py

To satisfy the system design requirements the following user interface sub-applications were developed:

Sub-Application Architecture Documentation

Each sub-application has detailed architecture documentation covering requirements, implementation status, data flow, components, testing, and recommendations:

1. Software Update Check - Checks for and downloads software updates

   • frontend_tkinter_software_update.py
   • data_model_software_updates.py

2. Flight Controller Communication - Establishes FC connection, downloads parameters and metadata

   • frontend_tkinter_connection_selection.py
     • frontend_tkinter_flightcontroller_info.py
     • backend_flightcontroller.py - Main facade class using delegation pattern
       • backend_flightcontroller_connection.py - Connection management
       • backend_flightcontroller_params.py - Parameter operations
       • backend_flightcontroller_commands.py - Command execution
       • backend_flightcontroller_files.py - File operations
       • data_model_flightcontroller_info.py - FC information
       • backend_flightcontroller_protocols.py - Protocol definitions
     • backend_mavftp.py

3. Directory and Project Selection - Creates new projects or opens existing ones

   • frontend_tkinter_directory_selection.py
     • frontend_tkinter_template_overview.py

4. Vehicle Component Editor - Defines vehicle components and their FC connections

   • frontend_tkinter_component_editor.py
     • frontend_tkinter_component_editor_base.py
     • frontend_tkinter_component_template_manager.py

5. Parameter Editor and Uploader - Views documentation, edits parameters, uploads to FC

   • sequentially for each configuration step:
     • view documentation relevant for the current configuration step,
     • edit parameters relevant for the current configuration step,
     • upload them to the flight controller,
     • save them to file
   • frontend_tkinter_parameter_editor.py
     • frontend_tkinter_parameter_editor_documentation_frame.py
     • frontend_tkinter_parameter_editor_table.py
     • frontend_tkinter_stage_progress.py

6. Motor Test - Provides motor testing functionality with visual diagram display

   • frontend_tkinter_motor_test.py
   • data_model_motor_test.py

Each sub-application can be run in isolation, so it is easier to test and develop them.

Data Models and Architecture

Each application separates the business logic (data_model_*.py) from the user interface logic (frontend_tkinter_*.py). This improves testability and maintainability of the code.

1. Check for software updates:
   1. data_model_software_updates.py
2. FC connection:
   1. data_model_fc_ids.py <- autogenerated by update_flight_controller_ids.py
3. Create or open project:
   1. data_model_vehicle_project.py
   2. data_model_vehicle_project_creator.py
   3. data_model_vehicle_project_opener.py
   4. data_model_template_overview.py
4. Component editor:
   1. battery_cell_voltages.py
   2. data_model_vehicle_components_base.py
   3. data_model_vehicle_components_display.py
   4. data_model_vehicle_components_import.py
   5. data_model_vehicle_components_json_schema.py
   6. data_model_vehicle_components_templates.py
   7. data_model_vehicle_components_validation.py
   8. data_model_vehicle_components.py
5. Parameter editor:
   1. data_model_configuration_step.py
   2. data_model_par_dict.py
   3. data_model_ardupilot_parameter.py

The detailed data models, components, and dependencies for each sub-application are documented in their respective architecture files linked above.

Common Libraries and Shared Components

All applications use one or more of the following shared libraries:

1. internationalization

   1. __init__.py
   2. internationalization.py
   3. configuration_steps_strings.py <- autogenerated by update_configuration_steps_translation.py
   4. vehicle_components.py <- autogenerated by update_vehicle_components_translation.py

2. command line parsing

   1. common_arguments.py
   2. argparse_check_range.py

3. the local filesystem backend does file I/O on the local file system. Operates mostly on parameter files and metadata/documentation files

   1. backend_filesystem.py
   2. backend_filesystem_vehicle_components.py
   3. backend_filesystem_configuration_steps.py
   4. backend_filesystem_program_settings.py

4. the internet backend communicates with the internet

   1. backend_internet.py

5. the flight controller backend communicates with the flight controller using a delegation pattern with specialized managers:

   1. backend_flightcontroller.py - Main facade class that delegates to specialized managers
   2. backend_flightcontroller_connection.py - Handles connection establishment, port discovery, and heartbeat detection
   3. backend_flightcontroller_params.py - Manages parameter download, upload, and querying
   4. backend_flightcontroller_commands.py - Executes MAVLink commands (motor tests, battery status, etc.)
   5. backend_flightcontroller_files.py - Handles file upload/download via MAVFTP
   6. data_model_flightcontroller_info.py - Stores flight controller metadata and capabilities
   7. backend_flightcontroller_protocols.py - Protocol definitions for dependency injection and testing
   8. backend_mavftp.py - MAVFTP protocol implementation

   Flight Controller Backend Architecture:

   • The FlightController class acts as a facade, delegating operations to specialized managers
   • Each manager handles a specific concern (connection, parameters, commands, files)
   • Managers can reference each other (e.g., params manager holds reference to connection manager)
   • Protocol definitions enable dependency injection for testing
   • Connection manager is the source of truth for connection state (master, comport, info)
   • Other managers query connection manager for current state rather than caching it

   Error Handling Standards:

   To maintain consistency across the flight controller backend, the following error handling patterns are used:

   1. Connection and I/O Operations - Return str (empty string on success, error message on failure):

      • connect(), disconnect(), register_and_try_connect()
      • Rationale: Allows user-friendly error messages to be displayed directly

   2. Command Operations - Return tuple[bool, str] (success flag, error message):

      • test_motor(), test_all_motors(), reset_all_parameters_to_default()
      • Rationale: Separates success/failure from error details for better control flow

   3. Query Operations - Return Optional[T] or raise exceptions:

      • fetch_param() - raises TimeoutError on timeout
      • get_battery_status() - returns tuple[Optional[tuple[float, float]], str]
      • Rationale: Distinguishes between "not found" (None) and "error" (exception)

   4. Bulk Operations - Return data structures or tuples:

      • download_params() - returns tuple[dict[str, float], ParDict]
      • Rationale: Success implied by returned data, errors logged but not returned

   Testing Hacks and Violations:

   The following methods exist for testing purposes only and violate architectural principles:

   • FlightController.set_master_for_testing() - Directly mutates connection manager's internal state, violating the principle that connection manager should be the sole mutator of connection state. Never use in production code.

   • Test parameter loading via device=DEVICE_FC_PARAM_FROM_FILE in backend_flightcontroller_params.py - Bypasses normal parameter download flow to load from local file.

6. the tkinter frontend, which is the GUI the user interacts with

   1. frontend_tkinter_autoresize_combobox.py
   2. frontend_tkinter_base_window.py
   3. frontend_tkinter_entry_dynamic.py
   4. frontend_tkinter_pair_tuple_combobox.py
   5. frontend_tkinter_progress_window.py
   6. frontend_tkinter_rich_text.py
   7. frontend_tkinter_scroll_frame.py
   8. frontend_tkinter_show.py
   9. frontend_tkinter_usage_popup_window.py

When all is combined it looks like this:

The parts can be individually tested, and do have unit tests. They can also be exchanged, for instance, tkinter-frontend can be replaced with wxWidgets or pyQt.

In the future, we might port the entire application into a client-based web application. That way the users would not need to install the software and will always use the latest version.

Generated Files

Several files in the project are automatically generated and should not be manually edited:

Core Application Generated Files

1. data_model_fc_ids.py <- autogenerated by update_flight_controller_ids.py

   • Contains flight controller hardware identification mappings
   • Generated from ArduPilot firmware metadata

2. configuration_steps_strings.py <- autogenerated by update_configuration_steps_translation.py

   • Contains translatable strings for configuration step descriptions
   • Generated from configuration JSON files for internationalization

3. vehicle_components.py <- autogenerated by update_vehicle_components_translation.py

   • Contains translatable strings for vehicle component descriptions
   • Generated from vehicle component JSON schemas for internationalization

Development and Analysis Tools

1. generate_codebase_pie_chart.py <- generated by AI assistant for codebase analysis

   • Creates pie chart visualizations of codebase structure
   • Automatically maintains codebase metrics and statistics
   • Part of the development workflow for project health assessment

2. download_motor_diagrams.py <- autogenerated by .github/instructions/update_motor_diagrams.md

   • Downloads motor diagram SVG files from ArduPilot documentation
   • Updates motor diagrams used in the motor test sub-application
   • Periodically updated via GitHub Actions workflow

Note: When calculating code metrics or performing codebase analysis, these generated files should be counted separately from hand-written code to provide accurate development statistics.

Module design

To assure code quality we decided to use Microsoft VS code with a lot of extensions to lint the code as you type. We use git pre-commit hooks to check the code before it is committed to the repository.

Module testing

We tested using automated static tests in both pre-commit hooks and on github CI:

• ruff
• pylint
• mypy
• pyright
• markdown-lint
• markdown-link-check
• codespell, grammarly
• shellcheck

We tested using automated dynamic tests on github CI including automated test coverage reports. We use unittest to write unit tests for the code. The tests are easy to run on the command line or in VS code. When you write new code you must also write tests in the tests/ directory, there is CI test that only passes if test coverage increases monotonically.

To run the tests either use the python tests plugin in visualstudio code, or execute:

pytest

Integration testing

The different sub-applications are first tested independently. Each has detailed testing strategies documented in their respective architecture files:

• Software Update - python .\ardupilot_methodic_configurator\data_model_software_updates.py
• Flight Controller Communication - python .\ardupilot_methodic_configurator\frontend_tkinter_connection_selection.py
• Directory Selection
  • python .\ardupilot_methodic_configurator\frontend_tkinter_directory_selection.py
  • python .\ardupilot_methodic_configurator\frontend_tkinter_template_overview.py
• Component Editor - python .\ardupilot_methodic_configurator\frontend_tkinter_component_editor.py
• Parameter Editor - python .\ardupilot_methodic_configurator\frontend_tkinter_parameter_editor.py
• Motor Test - python .\ardupilot_methodic_configurator\frontend_tkinter_motor_test.py

Only after each one performs 100% as expected, they are integrated and tested together. This speeds up the development process.

System testing

Here the integrated application was tested against the system requirements defined above. The tests were conducted on Windows and Linux machines using multiple different flight controllers and firmware versions. The software is automatically build and distributed using the github CD pipeline.

Acceptance testing

The software was tested by multiple users with different skill levels, languages, flight controller and backgrounds. The feedback was used to improve the user interface and user experience. The software is ready for production use since November 2024.

DevOps

The software is feature complete and stable with a user base of hundreds of users, we switched from the V-Model to DevOps development process on November 2024. This provides faster response to requirements changes and additions.

Do a release

The release process is automated. To do a release navigate to the bump_version_and_tag workflow and select Run Workflow enter the version and the description and press the green Run Workflow button.

<script async src="https://widget.gurubase.io/widget.latest.min.js" data-widget-id="uE4kxEE4LY3ZSyfNsF5bU6gIOnWGTBOL_e16KwDH-0g" data-text="Ask AI" data-margins='{"bottom": "1rem", "right": "1rem"}' data-light-mode="true" id="guru-widget-id"> </script>