Skip to content

fw: add Memfault SDK#3

Merged
gmarull merged 1 commit intocoredevices:mainfrom
noahp:memfault
Jun 18, 2025
Merged

fw: add Memfault SDK#3
gmarull merged 1 commit intocoredevices:mainfrom
noahp:memfault

Conversation

@noahp
Copy link
Contributor

@noahp noahp commented May 29, 2025

Summary

Add the Memfault SDK as a git submodule, and integrate it into the
project.

This enables tracking the following in Memfault:

  • reboots (translated pebble reboot reason codes)
  • heartbeat metrics

Some console commands are included to help test the integration:

  • mflt export : dump serialized Memfault chunks
  • mflt heartbeat : trigger a heartbeat
  • mflt metrics_dump : print current heartbeat values
  • mflt device_info : print device serial, software type, hardware
    version

TODO:

  • log capture
  • Memfault coredumps

Testing

Flashed to an asterix board, and ran these test commands:

mflt heartbeat
mflt export

Uploaded the symbol file (build/src/fw/tintin_fw.elf) and chunks to
app.memfault.com and confirmed both processed correctly.

Note: reboot reason tracking is not implemented on the nRF5 platform
(see reboot_reason.c). We'll need a solution there, eg just a small
slice of .noinit ram ideally.


Signed-off-by: Noah Pendleton noah@memfault.com

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@ericmigi
Copy link
Collaborator

Don't worry about the CLA for PebbleOS, I need to disable that

Copy link
Member

@gmarull gmarull left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initial comments: make sure to make this feature optional; otherwise we won't be able to merge back to the OSS repository.

@gmarull gmarull force-pushed the main branch 2 times, most recently from f7efcd6 to 55ac7aa Compare June 2, 2025 13:27
@noahp noahp force-pushed the memfault branch 2 times, most recently from 8cc05ec to 15cbf97 Compare June 4, 2025 15:23
@noahp
Copy link
Contributor Author

noahp commented Jun 4, 2025

Initial comments: make sure to make this feature optional; otherwise we won't be able to merge back to the OSS repository.

Thanks @gmarull , I added a ./waf configure --no-memfault option to support this! LMK if you had a different approach in mind 🙏

@gmarull
Copy link
Member

gmarull commented Jun 9, 2025

Initial comments: make sure to make this feature optional; otherwise we won't be able to merge back to the OSS repository.

Thanks @gmarull , I added a ./waf configure --no-memfault option to support this! LMK if you had a different approach in mind 🙏

Sorry to come late, I've been busy with other urgent stuff. I think memfault should probably be disabled by default, maybe enabled just on platforms we want it to be default (asterix/obelix). Another question: how is communication authenticated? Is fw just reporting logs to the mobile and mobile holds credentials?

@ericmigi
Copy link
Collaborator

@noahp how's things looking on this PR? Can we help in any way? We're aiming to start a beta test in 2-3 weeks. Any chance we might have coredumps and heart beats by then? We could do the wiring up for data for the heart beat

## Summary

Add the Memfault SDK as a git submodule, and integrate it into the
project.

This enables tracking the following in Memfault:

- reboots (translated pebble reboot reason codes)
- heartbeat metrics

Some console commands are included to help test the integration:

- `mflt export` : dump serialized Memfault chunks
- `mflt heartbeat` : trigger a heartbeat
- `mflt metrics_dump` : print current heartbeat values
- `mflt device_info` : print device serial, software type, hardware
  version

Memfault is enabled by default for `asterix` + `obelix`. Provide a
configure options, `--memfault {0,1}` to override the default.

TODO:

- log capture
- Memfault coredumps

## Testing

Flashed to an asterix board, and ran these test commands:

```bash
mflt heartbeat
mflt export
```

Uploaded the symbol file (`build/src/fw/tintin_fw.elf`) and chunks to
`app.memfault.com` and confirmed both processed correctly.

Note: reboot reason tracking is not implemented on the nRF5 platform
(see `reboot_reason.c`). We'll need a solution there, eg just a small
slice of .noinit ram ideally.

---

Signed-off-by: Noah Pendleton <noah@memfault.com>
@noahp noahp marked this pull request as ready for review June 16, 2025 19:50
@noahp
Copy link
Contributor Author

noahp commented Jun 16, 2025

Sorry to come late, I've been busy with other urgent stuff. I think memfault should probably be disabled by default, maybe enabled just on platforms we want it to be default (asterix/obelix). Another question: how is communication authenticated? Is fw just reporting logs to the mobile and mobile holds credentials?

Ok, I changed it to what you asked: enabled by default for asterix + obelix, otherwise opt-in.

how is communication authenticated? Is fw just reporting logs to the mobile and mobile holds credentials?

Yes, Memfault data is sent with a Project Key, which is typically stored on the data relay (mobile app in this system): https://api-docs.memfault.com/#a8d3e36f-62f0-4120-9fc6-544ee04f3bb5

This PR is ready for review. There are some remaining items:

  • enabling coredumps. my plan is to wholesale swap the Memfault fault handlers here, when Memfault is enabled. we will need to hook the flash API as a Memfault coredump storage backend. shouldn't be much work.
  • log instrumentation
  • update reboot reason tracking to work on the nRF52

Data can be exported for upload by calling memfault_packetizer_get_chunk(), cc @sjp4 who knows the drill here https://docs.memfault.com/docs/mcu/data-from-firmware-to-the-cloud#normal-mode

@noahp noahp requested a review from gmarull June 16, 2025 20:39
Comment on lines +17 to +21
# For Memfault build id, we want to make sure LTO does not force inline
# TODO for some reason, this is applying to jerryscript link step, which we don't want.
# conf.env.append_value(
# "LINKFLAGS", ["-Wl,--require-defined=g_memfault_build_id"]
# )
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

open GH issue to track

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cleaned up in this patch- #51

Comment on lines +43 to +44
# todo, figure out memfault support for FreeRTOS V8.2.1
# "memfault-firmware-sdk/ports/freertos/src/*.c",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

open github issue to track

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 added here, thank you!
#52

@gmarull gmarull merged commit 6eb00df into coredevices:main Jun 18, 2025
16 checks passed
noahp added a commit to noahp/pebbleos that referenced this pull request Jun 18, 2025
I ended up setting this in `src/fw/wscript` instead, so remove the dead
code.

See reference here:

coredevices#3 (comment)

Signed-off-by: Noah Pendleton <noah@memfault.com>
gmarull pushed a commit that referenced this pull request Jun 18, 2025
I ended up setting this in `src/fw/wscript` instead, so remove the dead
code.

See reference here:

#3 (comment)

Signed-off-by: Noah Pendleton <noah@memfault.com>
phoddie pushed a commit to phoddie/pebbleos that referenced this pull request Jul 1, 2025
I ended up setting this in `src/fw/wscript` instead, so remove the dead
code.

See reference here:

coredevices#3 (comment)

Signed-off-by: Noah Pendleton <noah@memfault.com>
jwise pushed a commit to jwise/pebble-firmware that referenced this pull request Nov 25, 2025
I ended up setting this in `src/fw/wscript` instead, so remove the dead
code.

See reference here:

coredevices/PebbleOS#3 (comment)

Signed-off-by: Noah Pendleton <noah@memfault.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants