How to recreate v1.3.3.20 binary in a local build

[mha1]

Hi,

I'm trying to recreate mm-stm-serial-aetr-v1.3.3.20.bin using

• source commit 2bd50f4 (= tag 1.3.3.20)
• local Arduino 1.8.16 install
• board package Multi-Module STM32 Boards (Devel) 1.3.1 (uses xpack-arm-none-eabi-gcc 8.3.1-1.4 tools)
• board Multi X-in-1 STM32F103CB 128kB

v1.3.3.20 asset mm-stm-serial-aetr-v1.3.3.20.bin has a size of 118,192 bytes

My local build with just PPM disabled in _Config.h produces a .bin with a size of 117,372 bytes

What am I missing? Is there anything I need to enable to recreate mm-stm-serial-aetr-v1.3.3.20.bin?

Thanks,
Michael

[benlye]

Are you building on Windows? We build our release files on Linux. A <1KB file size delta when building on Windows seems to be 'normal' (but is interesting).

[mha1]

Thanks for your response. Indeed, I do build on Window. I can't say for sure but working on 1.3.3.7 and 1.3.3.14 my first actions was to recreate the baseline to make sure the tool chain in working. Think I managed to build digital twins of both. This was on the old gcc version.

Can you confirm //#define ENABLE_PPM is the only thing to change in _Config.h to build mm-stm-serial-aetr-v1.3.3.20.bin?

[benlye]

Can you confirm //#define ENABLE_PPM is the only thing to change in _Config.h to build mm-stm-serial-aetr-v1.3.3.20.bin?

Yes, that's all you need to do.

[mha1]

I remember the switch to the new gcc version was done to save flash memory. Just switch to building on Windows and get another 800 bytes free of charge.

[mha1]

The best way to do "genuine" builds would be to use CI. Unfortunately it seems running actions off of my fork excludes release type builds. Is there any way to do a release type cloud build? Alternatively a docker would also be fine. Possible?

[benlye]

In your repo you can modify the existing CI workflow, or create your own. Personally, I use WSLv2 to build binary files on Windows. The output is the same as the Github CI process.

[mha1]

Care to share a how-to-setup?

[mha1]

My bad. Running CI on my local repo does build release type .bins and it produces binary twins.

But I'd be still interested how you build on WSL2. Would MSYS2 work too?

[benlye]

I've uploaded a copy of my WSLv2 build script here: https://gist.github.com/benlye/1b33b6c2bbf9883f52fbdf81148105d7

It's basically just cobbled together from the code we use for our CI/CD workflows. It 'should' just work, and will install all the necessary bits you need in your Linux environment. I use it in an Ubuntu 22.04 image running in WSLv2 on Windows 11.

It builds 10 different images for me, each with slightly different configuration. It should be pretty easy for you to customize to do whatever you want. At a minimum you will need to change SRCPATH and you probably want to remove the lines which set the fixed global ID options, FORCE_GLOBAL_ID and FORCE_CYRF_ID.

I don't if it will work with MSYS2 - I've never tried.

[mha1]

Great, I'll try it.

[benlye]

I've also realized that I did create a Docker image some time ago, but it's rather out of date now. I'll see if I can get it updated.

[mha1]

Thanks for your script. Works like a charm on WSL2/Ubuntu 20.04. Finally time to scrap the Arduino IDE.

[benlye]

I've updated my Docker container if you want to give that a try.
https://hub.docker.com/repository/docker/benlye/multi4in1-build/

[mha1]

Hi,

great, docker works too. I created binary identical V1.3.3.20 stm32-serial builds but the documentation needs one (maybe two) updates.

docker run --rm -it -v [Firmware Source Path]:/multifw -e BOARD=[board] benlye/multi4in1-build asks for [Firmware Source Path]:/multifw but the examples use [Firmware Source Path]:/multi

Both Linux and Windows examples use [Firmware Source Path]\Multiprotocol:/multi:. This results in ERROR: Multiprotocol source not found.. It works with leaving out \Multiprotocol

I think the docker solution should make it to the web page.

Thanks again,
Michael

[benlye]

I've put some more time into the Docker container, including cleaning up the inconsistencies in the documentation.

• The container is about 25% smaller than it was
• You can use [Source Path]\Multiprotocol or just [Source Path] in the mount point and it will work
• I added an environment variable to specify a custom name for bin file
• Made the output a bit more useful
• Added lots of testing to the CI process

I'll get instructions for building with the container added to the main MULTI documentation.

Ben

C:\Users\blye\> docker run --rm -it benlye/multi4in1-build

Multi4in1-Build - https://hub.docker.com/r/benlye/multi4in1-build

ERROR: Multiprotocol source not found. Did you pass the '-v' option correctly?

Usage:
  docker run --rm -it -v "[path to source]:/multi" -e "BOARD=[board]" [-e "FWBINNAME=[output file name]"] benlye/multi4in1-build

Where:
  [path to source] is the path to the folder where the Multiprotocol source was cloned or unzipped
  [board] case-sensitive FQBN or alias of the desired board
  [output file name] optional name for the compiled firmware file (default naming is used if not specified)

Valid boards:
  Type        | Alias           | FQBN
  ------------|-----------------|------------------------------------------------------
  AVR         | avr             | multi4in1:avr:multiatmega328p:bootloader=none
  AVR         | avr-optiboot    | multi4in1:avr:multiatmega328p:bootloader=optiboot
  STM32 128KB | stm32           | multi4in1:STM32F1:multistm32f103cb:debug_option=none
  STM32 128KB | stm32-usbdebug  | multi4in1:STM32F1:multistm32f103cb:debug_option=native
  STM32 128KB | stm32-ftdidebug | multi4in1:STM32F1:multistm32f103cb:debug_option=ftdi
  STM32 64KB  | stm32-64        | multi4in1:STM32F1:multistm32f103c8:debug_option=none
  T18 5in1    | t18-5in1        | multi4in1:STM32F1:multi5in1t18int
  OrangeRX    | orx             | multi4in1:avr:multixmega32d4

Notes:
  1. Output file name can include the text '{VERSION}' which will be automatically replaced by the firmware version number
  2. 'Devel' boards can be specified by appending '-devel' to the alias (e.g. 'stm32-devel') or replacing 'multi' with 'multi-devel' in the FQBN.

Examples:
  docker run --rm -it -v "C:\Users\benlye\Downloads\DIY-Multiprotocol-TX-Module:/multi" -e "BOARD=stm32" benlye/multi4in1-build
  docker run --rm -it -v "C:\Users\benlye\Downloads\DIY-Multiprotocol-TX-Module:/multi" -e "BOARD=multi4in1:STM32F1:multistm32f103cb:debug_option=native" benlye/multi4in1-build
  docker run --rm -it -v "C:\Users\benlye\Downloads\DIY-Multiprotocol-TX-Module:/multi" -e "BOARD=stm32" -"FWBINFILE=myfirmware.bin" benlye/multi4in1-build
  docker run --rm -it -v "C:\Users\benlye\Downloads\DIY-Multiprotocol-TX-Module:/multi" -e "BOARD=stm32" -"FWBINFILE=myfirmware-{VERSION}.bin" benlye/multi4in1-build

C:\Users\blye> docker run --rm -it -v 'C:\Users\blye\Github\DIY-Multiprotocol-TX-Module/:/multi' -e 'BOARD=stm32' benlye/multi4in1-build

Multi4in1-Build - https://hub.docker.com/r/benlye/multi4in1-build

Checking for Multi4in1 board package updates ...
All the cores are already at the latest version

Building firmware version v1.3.3.20 using multi4in1:STM32F1:multistm32f103cb:debug_option=none...

arduino-cli compile -b multi4in1:STM32F1:multistm32f103cb:debug_option=none /tmp/build/Multiprotocol/Multiprotocol.ino --build-path /build/output

Sketch uses 119468 bytes (98%) of program storage space. Maximum is 120808 bytes.
Global variables use 4464 bytes (21%) of dynamic memory, leaving 16016 bytes for local variables. Maximum is 20480 bytes.

Used library Version Path
SPI          1.0     /root/.arduino15/packages/multi4in1/hardware/STM32F1/1.2.2/libraries/SPI
EEPROM               /root/.arduino15/packages/multi4in1/hardware/STM32F1/1.2.2/libraries/EEPROM

Used platform     Version Path
multi4in1:STM32F1 1.2.2   /root/.arduino15/packages/multi4in1/hardware/STM32F1/1.2.2

SUCCESS: Compiled Multiprotocol firmware saved as '/multi/Multiprotocol/multi-stm-1.3.3.20.bin'.
         (The firmware file can be found in the same location as 'Multiprotocol.ino on the Docker host')

[mha1]

Hi Ben,

Thank you very much. I tried with the updated container (d16b0cf8d6f3ef4d492bbd5499760f620d959d234656a84d9d3945e6683169b2):

• the documentation on https://hub.docker.com/r/benlye/multi4in1-build is still the old one, isn't it?
• adding -"FWBINFILE=myfirmware.bin" doesn't work for me. I get this error message: unknown shorthand flag: 'F' in -FWBINFILE=myfirmware.bin

Michael

[benlye]

I've synced the Docker Hub doc up with Github. To use FWBINFILE you need -e before it because it's an environment variable:
-e "FWBINFILE=myfirmware.bin"

[mha1]

Hi Ben,

Yes, I was just copying your examples. The e's in -e are missing:

  docker run --rm -it -v "C:\Users\benlye\Downloads\DIY-Multiprotocol-TX-Module:/multi" -e "BOARD=stm32" -"FWBINFILE=myfirmware.bin" benlye/multi4in1-build
  docker run --rm -it -v "C:\Users\benlye\Downloads\DIY-Multiprotocol-TX-Module:/multi" -e "BOARD=stm32" -"FWBINFILE=myfirmware-{VERSION}.bin" benlye/multi4in1-build

But I still can't get FWBINFILE to work. It doesn't seem to use the name set with FWBINFILE. Check this out:

docker run --rm -it -v "C:\Users\Mickey\Desktop\LocalGit\DIY-Multiprotocol-TX-Module\:/multi" -e "BOARD=stm32-devel" -e "FWBINFILE=myfirmware.bin" benlye/multi4in1-build

Multi4in1-Build - https://hub.docker.com/r/benlye/multi4in1-build

Checking for Multi4in1 board package updates ...
All the cores are already at the latest version

WARNING: Using 'devel' board package.

Building firmware version v1.3.3.20 using multi4in1-devel:STM32F1:multistm32f103cb:debug_option=none...

arduino-cli compile -b multi4in1-devel:STM32F1:multistm32f103cb:debug_option=none /tmp/build/Multiprotocol/Multiprotocol.ino --build-path /build/output

Sketch uses 118168 bytes (97%) of program storage space. Maximum is 120808 bytes.
Global variables use 4480 bytes (21%) of dynamic memory, leaving 16000 bytes for local variables. Maximum is 20480 bytes.

Used library Version Path
SPI          1.0     /root/.arduino15/packages/multi4in1-devel/hardware/STM32F1/1.3.1/libraries/SPI
EEPROM               /root/.arduino15/packages/multi4in1-devel/hardware/STM32F1/1.3.1/libraries/EEPROM

Used platform           Version Path
multi4in1-devel:STM32F1 1.3.1   /root/.arduino15/packages/multi4in1-devel/hardware/STM32F1/1.3.1

SUCCESS: Compiled Multiprotocol firmware saved as '/multi/Multiprotocol/multi-stm-1.3.3.20.bin'.
         (The firmware file can be found in the same location as 'Multiprotocol.ino on the Docker host')

[benlye]

OK, got it - the 'help' output from the container was giving bad examples and had the wrong variable name. It should be FWBINNAME not FWBINFILE. I've fixed it and the new container is deploying now. The only change is to the help text so if you try the current version with -e "FWBINNAME=myfirmware.bin" it should work.

Multi4in1-Build - https://hub.docker.com/r/benlye/multi4in1-build

Usage:
  docker run --rm -it -v "[path to source]:/multi" -e "BOARD=[board]" [-e "FWBINNAME=[output file name]"] benlye/multi4in1-build

Where:
  [path to source] is the path to the folder where the Multiprotocol source was cloned or unzipped
  [board] case-sensitive FQBN or alias of the desired board
  [output file name] optional name for the compiled firmware file (default naming is used if not specified)

Valid boards:
  Type        | Alias           | FQBN
  ------------|-----------------|------------------------------------------------------
  AVR         | avr             | multi4in1:avr:multiatmega328p:bootloader=none
  AVR         | avr-optiboot    | multi4in1:avr:multiatmega328p:bootloader=optiboot
  STM32 128KB | stm32           | multi4in1:STM32F1:multistm32f103cb:debug_option=none
  STM32 128KB | stm32-usbdebug  | multi4in1:STM32F1:multistm32f103cb:debug_option=native
  STM32 128KB | stm32-ftdidebug | multi4in1:STM32F1:multistm32f103cb:debug_option=ftdi
  STM32 64KB  | stm32-64        | multi4in1:STM32F1:multistm32f103c8:debug_option=none
  T18 5in1    | t18-5in1        | multi4in1:STM32F1:multi5in1t18int
  OrangeRX    | orx             | multi4in1:avr:multixmega32d4

Notes:
  1. Output file name can include the text '{VERSION}' which will be automatically replaced by the firmware version number
  2. 'Devel' boards can be specified by appending '-devel' to the alias (e.g. 'stm32-devel') or replacing 'multi' with 'multi-devel' in the FQBN.

Examples:
  docker run --rm -it -v "C:\Users\benlye\Downloads\DIY-Multiprotocol-TX-Module:/multi" -e "BOARD=stm32" benlye/multi4in1-build
  docker run --rm -it -v "C:\Users\benlye\Downloads\DIY-Multiprotocol-TX-Module:/multi" -e "BOARD=multi4in1:STM32F1:multistm32f103cb:debug_option=native" benlye/multi4in1-build
  docker run --rm -it -v "C:\Users\benlye\Downloads\DIY-Multiprotocol-TX-Module:/multi" -e "BOARD=stm32" -e "FWBINNAME=myfirmware.bin" benlye/multi4in1-build
  docker run --rm -it -v "C:\Users\benlye\Downloads\DIY-Multiprotocol-TX-Module:/multi" -e "BOARD=stm32" -e "FWBINNAME=myfirmware-{VERSION}.bin" benlye/multi4in1-build

[mha1]

Hi Ben,

Great, thank you very much. Works perfect now!

Maybe on thing. Using -devel on the alias outputs a warning WARNING: Using 'devel' board package.. Warning sounds like something is out of the extraordinary yet -devel is the "official" way now to build 1.3.3.20 and following. I'd output something like this:

non -devel:
Using gcc verison 4.8.3-2014 (recommended for STM32 builds 1.3.3.14 and older)

-devel:
Using gcc version 8.3.1-14 (recommended for STM32 builds after 1.3.3.14)

Have a great day

Michael

[benlye]

My preferred approach will be to promote the changes we've made in the devel boards package to the non-devel version. I need to talk to @pascallanger about doing that.

[mha1]

Makes sense if there is sufficient evidence -devel is working. Official 1.3.3.20 builds (latest) were done with -devel, weren't they? So if there is no major outcry so far it's probably ok and a good idea to make -devel a production version. I at least flew 1.3.3.20 with no problems. And for backwards compatibility (e.g. to recreate 1.3.3.14) allow using the old version.

[pascallanger]

@benlye please go ahead, thanks!

[benlye]

This is done - the non-devel STM32 board package is now v1.3.1. I'm updating the Docker container so that it will have the new version of the non-devel board, this should be deployed in about 10 minutes.

@pascallanger at some point in the next few days I'll update the main CI/CD workflows to switch back to the non-devel board, so that we can keep the 'devel' version for board development/changes in the future.

[mha1]

Hi Ben,

I tried the new container and guess at this point it is ok devel and non-devel produce the same results as in devel there are no additions to 1.3.1 yet. One point maybe. It is now no longer possible to reproduce 1.3.3.14 and earlier versions as using non-devel uses gcc-8.3.1.

Have a grat day

Michael