SRM Notes for 27-03-2025

[bruno-f-cruz]

Agenda

• 🚀Small Issue (10 min)
• 🚅Medium Issue (20min)
• 🐢Large Issue (45min)

Reviewing

• Resuming the discussion on versioning and SDKs 🐢 :
  • Add semantic version register #68
  • Add compatiblity table between harp hardware core implementations #125
  • Refactor Version registers #44
  • Add implementation information to device schema #43

[bruno-f-cruz]

Notes from SRM

In the device.yml

• Remove firmwareVersion and hardwareTargets. These are currently not being used for any interface and should be ok to outright get rid of
• Add a new field InterfaceVersion that versions the device.yml as a specification of the interface of the device. This is not to be confused with the firmware version. Firmware can be updated without leading to interface changes

In the core.yml

• Rename protocolVersion to protocolInterfaceVersion. This will version the interface of the core register and should be shared across all devices (regardless of underlying SDK).

On Optional registers (#122)

• All register in the core MUST be available if specified in the corresponding version of the core.yml interface
• The notion of optional refers to register functionality.
• For instance, an optional register TAG MUST be implemented by exposing a register that can be read from. However the value of such register is allowed to default to 0, as specified in the spec. In the future, we should try to keep these Optional registers to a minimum and document the SDK-specific behavior in the corresponding core repository/readme/website documentation.
• This behavior also applies to register deprecation. Once a register is deprecated, but still in the core.yml for backwards compatibility, the bleeding edge implementation is expected to honor the interface until the register is fully removed from the spec.

On the new version register (#68)

The proposal should be updated to:

• Add the InterfaceVersion as part of the specification (which now removes the previous PROTOCOL :protocol, 9, 11 line

   section Id
   HARDWARE        :hardware, 0, 3
   CORE                  :core, 3, 6
   FIRMWARE        :firmware, 6, 9
   INTERFACE_VERSION        :interface_version, 9, 11
   SDK             :architecture, 11, 12

Does the interface version need a patch version? I don't think so... @glopesdev @PathogenDavid @Poofjunior ?
Change SDK to a 4 byte value. For now allow arbitrary byte values but later use a string interface to parse it to some sort of human readable tag (e.g. pico)

Regarding Add implementation information to device schema

We decided to hold off on adding this information to the device.yml for now. If interfaces are stable this should not be a problem for any of the tools we currently support. We agreed that information on what core the device implements is instead better suited to be kept in the repository as a separate file or even in the README.

[bruno-f-cruz]

Adding a quick draft of the "onion" diagram (well it ended up as something else I guess...) that we floated around during the meeting. We can iterate on it during the next meeting.

@glopesdev raises a good point that everything becomes much simpler if we assume that there is only a single core.yml/core interface version that is shared among all devices.

[bruno-f-cruz]

the name protocolVersion could been seen as the release of the protocol repository. This includes yml schemas, device spec, protocol spec, etc...

• Should this be a SemVer? ou just a hash-like id?
• This only works if the notion of Optional is taken as meaning that all devices must implement the register interfaces, even if the underlying functionality is not available.

---

In the version register. Given this decision, it makes more sense to return the register name to protocolVersion.

---

What needs to go into the device.yml?

• protocolVersion (to xref against the protocol repository)

(Later)

• Tag register could be extended to be variable length (maybe just pad?) and just dump there whatever you want

What goes into the R_Version register?
firmwareVersion should be locked to the yml version
firmwareVersion patch number should NOT trigger a new device.yml.

[bruno-f-cruz]

After completion of :

• Add "under development" disclaimer to 32bit protocol #134
• Move WhoAmI table to a different repository #135
  The consensus was that the protocolVersion versions the main branch of the Protocol repository (https://github.com/harp-tech/protocol/)

[PathogenDavid]

😶 = Package existed but no version was released for it.

Only release packages when there's a change: (We probably won't do this)

Firmware Package	Interface Package	Python Package	Event
1.0.0	1.0.0		Initial release
1.0.1	😶		Firmware bug fix
😶	1.0.2		Interface regenerated with new generator
1.0.3	1.0.3		New register added
😶	1.0.4		device.yml change with no firmware implications
😶	😶	1.0.5	Python package introduced
😶	1.0.6	😶	C#-specific change
1.1.7	1.1.7	(lags)	New register, Python package is external and lags behind
😶	😶	1.1.7	Python maintainer catches up
1.1.8	😶	😶	Bug fix in firmware

Firmware forces interface release: (We will probably do this)

Firmware Package	Interface Package	Python Package	Event
1.0.0	1.0.0		Initial release
1.0.1	😶1.0.1		Firmware bug fix
😶	1.0.2		Interface regenerated with new generator
1.0.3	1.0.3		New register added
😶	1.0.4		device.yml change with no firmware implications
😶	😶1.0.5	1.0.5	Python package introduced
😶	1.0.6	😶	C#-specific change
1.1.7	1.1.7	(lags)	New register, Python package is external and lags behind
😶	😶	1.1.7	Python maintainer catches up
1.1.8	😶1.1.8	😶1.1.8	Bug fix in firmware

[PathogenDavid]

Other things we discussed (apologies if I forgot anything):

• We will use CI to automate most of this based on GitHub releases
• Firmware and interface packages will be versioned together, but firmware will be allowed to skip versions when no firmware change is needed. (Second table above)
• Releasing a firmware should always cause an interface release, but releasing an interface does not necessarily cause a firmware release.
  • Interface releases are "cheap" compared to firmware releases and this makes it easier to tell which interface version should be used for a given firmware version. (As noted below)
  • For in-band generated interfaces (IE: same repo as firmware) this is done automatically by CI.
  • For out-of-band generated interfaces (IE: provided by another repo, the Python example above) could be allowed to lag behind, but should strive to release every time there's a firmware release.
    • (Being in a separate repo doesn't necessarily force this to be the case, we can cause firmware releases to trigger releases in other interface repos. We do this for updating the Bonsai documentation repo when we release Bonsai.)
    • (Out-of-band interfaces aren't expected to be common, just mentioning this explicitly as an escape hatch.)
• Firmware primarily concerns the code burned into the device, but firmware packages could contain then-current device.yml for posterity.
• If you have a firmware version 1.0.3 and you want to grab a compatible interface, you can be confident knowing interface version 1.0.3 is compatible and available (if it's in-band.)
• If you have an interface version 1.0.6 and you want to know the preferred firmware, it'll be the version <= to the interface version. (So in the example table above, it'd be firmware 1.0.3)
• Interfaces can release whenever they want, but don't intrinsically cause a firmware release.
  • For example, changing an array of u32 to a struct would not affect the firmware
  • This is OK because using an old interface would not affect the ability to read the data, just how it's represented. (EG: If you used the u32[] version of the interface, the data is still read just fine. You just don't get the structure.)

Handling firmware blobs for devices with multiple hardware revisions

• For handling multiple hardware revisions (example) we currently publish separate hex files for each board revision. We could use a single UF2 file to distribute the separate firmwares. UF2 has provisions for embedding multiple firmware blobs associated with different family IDs.
  • Family IDs are registered here (registration is optional, but encouraged.) We could either allocate a handful of generic family IDs for Harp boards+revisions (IE: family IDs do not include whoami) or allocate a family ID for each revision of each Harp device (IE: family IDs do include whoami)
  • I also mentioned using a dedicated family ID to embed Harp-specific metadata in the UF2 blob
  • However what I forgot about is there's actually a provision for extension tags in the UF2 spec. We could/should use that instead for things like whoami and hardware revision disambiguation.
    • This would mean we could just use a single "Harp Device" family ID.
    • We might not even need our own family ID.
    • In the case of the Pico, the Pico has well-defined conventions for embedding metadata in the firmware so we might only use this for ATXMEGA devices. (Need to ponder this more)
• We could easily add the ability to extract raw firmware blobs from the UF2. (EG: Extract a single Behavior board UF2 into the three corresponding raw hex blobs. Or for Pico devices strip the Harp metadata and family ID.)
• Ditto on the opposite

[PathogenDavid]

• 🧶🧶🧶I don't think we explicitly discussed this, but these versions would be semver-like (specifically when it comes to firmware versions) but not strictly semver.
  • Strict semver would mean needing to bump the interface's major version when a breaking change occurs (EG: the u32[] to struct change would be breaking from that perspective.) However this becomes messy with multiple interfaces (for example, a device.yml change might be breaking for C# but not for Python. It could be breaking for one C# generator but not another.)
  • Maybe another way to word this: The firmware version is semver(ish), but the interface version is not.
  • I don't think there's a way out of this except versioning the interfaces separately which we've already decided is undesirable.
  • As I've mentioned before, I don't think strict semver is especially valuable for Harp and isn't something we should impose on firmware developers.
  • I vote for everyone to be OK with this because I think we're all getting kinda sick of talking about versions during Harp Club 🫠 (if this becomes a problem later we can always revisit it.)

---

We could just allow the interface to bump the major/minor version of the firmware, but this creates situations like this which are a bit odd:

Firmware Package	Interface Package	Event
1.0.0	1.0.0	Initial release
😶	2.0.0	A u32[] is changed to a struct
2.0.1	2.0.1	A minor bug is fixed in the firmware

Interface 2.0.0 preferring firmware 1.0.0 is weird when they otherwise tend to match. The 1.0.0 -> 2.0.1 jump for the firmware when all that changed was fixing a non-breaking bug is weird.