Thoughts on build process

[robertlipe]

It's not a secret how much I hate our build process. We rebuild WAY too much source (which we download and install for each of our ~50 targets) when there is any change in the build flags. (And when there's not, but that's not the point of today's sermon.)

build_src_flags allows one to set flags (optimizations, defines, error levels, etc.) that are specific to code in src/ - the code we care most about.

There's simply no reason to rebuild all of ArduinoFFT or (dear God) that stupid remote library just because we changed a GPIO pin or something. We probably DO have to recompile all of our own code - at least until we can localize some of that, but that's a future project - but it's comparatively small compared to our 3rdparty payload.

It seems we really have two groups of flags we carry around - the ones that ARE required for the world. S3 vs. C3, for example. Those clearly need to be in build_flags. But let's give some thought to moving all of "our" flags - the stuff that triggers what we're building, the pinouts of those displays, etc. down into "just" our own build.

Then, if you can imagine a functional caching build system (e.g. not the one that comes with PlatformIO but maybe something like ccache), maybe it can use that code 50 times instead of having to rebuild everything customized because the flags are slightly different in places they shouldn't even be exposed.

Just changing build_flags to build_src_flags and rebuilding demo, for example, I can see ir_Argo.cpp.o being built with ...

-DPLATFORMIO=60118 -DARDUINO_ESP32_DEV -DHAVE_CONFIG_H -DMBEDTLS_CONFIG_FILE="mbedtls/esp_config.h" -DUNITY_INCLUDE_CONFIG_H -DWITH_POSIX -D_GNU_SOURCE -DIDF_VER="v4.4.7-dirty" -DESP_PLATFORM -D_POSIX_READER_WRITER_LOCKS -DARDUINO_ARCH_ESP32 -DESP32 -DF_CPU=240000000L -DARDUINO=10812 -DARDUINO_VARIANT="esp32" "-DARDUINO_BOARD="Espressif ESP32 Dev Module"" -DARDUINO_PARTITION_partitions_custom

while, say, jsonserializer.cpp.o is built with all those PLUS -DDEMO=1. So, in theory, once the build system is sent to visit The Wizard (hint: it's not a heart that it needs), it should be able to reuse the display code and such across builds.

As a bonus, I just changed a comment in config.h, and it didn't recompile the whole world, just our world, which is at least understandable.

The changes really seem to be simple and would take the form of:
--- a/platformio.ini
+++ b/platformio.ini
@@ -356,7 +356,7 @@ board_build.embed_files = assets/bmp/brokenclouds.jpg

[env:demo]
extends = dev_esp32
-build_flags = -DDEMO=1
+build_src_flags = -DDEMO=1
${dev_esp32.build_flags}

; This is the basic DEMO project again, but expanded [ ... ]

Please join me in thinking about how well this might fit our world.

Pssst. sscache is apparently a better ccache than ccache is.