LPT: taming build caches

[GPSBabelDeveloper]

My anti-love for platformio's build system only grows over time. I was configuring backups on a machine where I routinely build ALL the ND projects¹ and noticed that, by far, my disk usage was blown out by NightDriver. WAT? NightDriver is about 20KLOC. But, of course, PlatformIO checks out every library we use ... 41 times (totaling about 6.5GB) and then proceeds to build it...41 times.

du -hs .pio/libdeps .pio/build_cache
5.9G	.pio/libdeps
174M	.pio/build_cache

So, my working directories (and I have a couple of these...) have about 174MB of caches.

By default, platformio caches this in ~/.platformio. I moved it some time ago so that it was per-project to respect my multiple projects. I've decided that I'm going to override this, but I don't want to patch .platformio.ini for each of my work trees or else we'll fight merge artifacts forever.

I found a basic recipe that can improve this. Sweeten to taste.

Time Machine in MacOS doesn't backup ~/Library/Caches on the premise that anything that goes in there should be reproducible upon demand. So I did the approximate equivalent of:

mkdir -p  ~/Library/Caches/platformio/build_cache
echo export PLATFORMIO_CACHE_DIR=~/Library/Caches/platformio/build_cache >> ~/.login

oh, and a rm -fr .pio/build_cache

It's tempting to set PLATFORMIO_LIBDEPS_DIR but I think that would work legitimately badly. Yesterday's example of picking up a new dependency would be a great example. Just because I updated one project doesn't mean I'm willing to accept that I'm
ready for a risky new FastLED (or whatever) in all.

Sweeten to taste for your own OS and system administration needs, of course. Just take the idea that these https://docs.platformio.org/en/latest/envvars.html#envvar-PLATFORMIO_BUILD_DIR override what's in platformio.ini

Now if PlatformIO acted like a sensible native program, it would know about things like where to put cache directories so that they can be shared when they should be and not backed up when they don't need to be, but that's not the world we live in.

That's today's Lipe Pro Tip.

¹ Granted, I'm probably one of three people that ever does this...