Skip to content

Make it possible to write tests #1059

@aqua

Description

@aqua

The Meshcore firmware build currently has no unit testing support. Testing embedded software is never trivial, but there's a lot of logic in MC that has no direct dependency on hardware and where development could be sped up with a test cycle that doesn't require flashing a device (routing, text/string manipulation, the CLIs, use of crypto, etc.) For example, I was recently interested in trying a change to the repeater advert timing logic to help make repeaters easier to discover in a newly-started region; it's the kind of thing I'd normally want to approach by getting the state machine right with a test harness and fake clocks before going to real hardware, and realized I needed to do this first.

People have a range of opinions on TDD and I'm not arguing that there should be a community shift towards it, but lowering barriers to testing always does encourage its use.

I have a working example which fixes all the current compatibility blockers to using PlatformIO's testing features and adds sample tests covering most of mesh::Util. It compiles nearly 100% of the non-platform-specific code (it currently only omits AutoDiscoverRTCClock, which I think is fixable) on env:native.

Notable changes required:

  • platformio configuration itself (nothing special)
  • a handful of fixes to code errors that the cross compiler was accepting but native-gcc won't (e.g. C99 variable-length array initializing, type mismatches, missing includes)
  • a couple minor tweaks to use of Arduino method that are inconsistent between the platform and native versions
  • mock Wire and SPI classes, which don't have mocks in the platformio registry; they're actually more like fakes currently, can be turned into mocks to test code that directly depends on their behavior)

I went with the googletest framework rather than Unity, in part because to my eyes it's easier to remember the matchers, and in part because the MC firmware doesn't have much logic/HAL separation and so having the mocking support available seems more important.

sample invocation:

aqua@cherry meshcore % pio test -e native
Verbosity level can be increased via `-v, -vv, or -vvv` option
Collected 2 tests

Processing test_native in native environment
------------------------------------------------------------------------------------------------------------------------------------------------------
Building...
Testing...
NopTest.ShouldPass	[PASSED]
--------------------------------------------------- native:test_native [PASSED] Took 1.77 seconds ---------------------------------------------------

Processing test_common in native environment
------------------------------------------------------------------------------------------------------------------------------------------------------
Building...
Testing...
UtilTests.NopTest	[PASSED]
UtilTests.SHA256	[PASSED]
UtilTests.toHex	[PASSED]
UtilTests.fromHex	[PASSED]
UtilTests.fromHexWrongSize	[PASSED]
UtilTests.isHexChar	[PASSED]
UtilTests.parseTextParts	[PASSED]
UtilTests.printHex	[PASSED]
--------------------------------------------------- native:test_common [PASSED] Took 2.06 seconds ---------------------------------------------------

====================================================================== SUMMARY ======================================================================
Environment    Test         Status    Duration
-------------  -----------  --------  ------------
native         test_native  PASSED    00:00:01.767
native         test_common  PASSED    00:00:02.060
===================================================== 9 test cases: 9 succeeded in 00:00:03.827 =====================================================

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions