build fails due to missing resources

[IzzySoft]

Trying to build the app using ./gradlew assembleRelease on Debian bookworm with JDK 17 fails, as some resources seem to be missing:

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:processReleaseResources'.
> A failure occurred while executing com.android.build.gradle.internal.res.LinkApplicationAndroidResourcesTask$TaskAction
   > Android resource linking failed
     ERROR: /build/repo/app/build/intermediates/incremental/release/mergeReleaseResources/stripped.dir/layout/baby_manager.xml:43: AAPT: error: resource drawable/pkg_not_wet (aka eu.pkgsoftware.babybuddywidgets:drawable/pkg_not_wet) not found.

     ERROR: /build/repo/app/build/intermediates/incremental/release/mergeReleaseResources/stripped.dir/layout/baby_manager.xml:55: AAPT: error: resource drawable/pkg_wet (aka eu.pkgsoftware.babybuddywidgets:drawable/pkg_wet) not found.

     ERROR: /build/repo/app/build/intermediates/incremental/release/mergeReleaseResources/stripped.dir/layout/baby_manager.xml:65: AAPT: error: resource drawable/pkg_no_poop (aka eu.pkgsoftware.babybuddywidgets:drawable/pkg_no_poop) not found.

     ERROR: /build/repo/app/build/intermediates/incremental/release/mergeReleaseResources/stripped.dir/layout/baby_manager.xml:77: AAPT: error: resource drawable/pkg_poop (aka eu.pkgsoftware.babybuddywidgets:drawable/pkg_poop) not found.

     ERROR: /build/repo/app/build/intermediates/incremental/release/mergeReleaseResources/stripped.dir/layout/baby_manager.xml:89: AAPT: error: resource drawable/pkg_notes (aka eu.pkgsoftware.babybuddywidgets:drawable/pkg_notes) not found.

     ERROR: /build/repo/app/build/intermediates/incremental/release/mergeReleaseResources/stripped.dir/layout/baby_manager.xml:100: AAPT: error: resource drawable/pkg_no_notes (aka eu.pkgsoftware.babybuddywidgets:drawable/pkg_no_notes) not found.

     ERROR: /build/repo/app/build/intermediates/incremental/release/mergeReleaseResources/stripped.dir/layout/quick_timer_entry.xml:79: AAPT: error: resource drawable/pkg_no_notes (aka eu.pkgsoftware.babybuddywidgets:drawable/pkg_no_notes) not found.

     ERROR: /build/repo/app/build/intermediates/incremental/release/mergeReleaseResources/stripped.dir/layout/quick_timer_entry.xml:91: AAPT: error: resource drawable/pkg_notes (aka eu.pkgsoftware.babybuddywidgets:drawable/pkg_notes) not
found.

     ERROR: /build/repo/app/build/intermediates/incremental/release/mergeReleaseResources/stripped.dir/layout/timeline_item.xml:66: AAPT: error: resource drawable/pkg_notes (aka eu.pkgsoftware.babybuddywidgets:drawable/pkg_notes) not found.

Could it be you forgot to check those in to this repo?

[MrApplejuice]

Likely the buildsystem is somewhat borked. Very likely in fact. I did not do a "from scratch" build in ages. I really really should get my build system in order ... 👀

Anyway, one problem is are the "non-free images" I am using from https://www.flaticon.com/ . I cannot add them to this repo according to their license. Therefore, one integral part of getting everything up and running is to get an API key from them so that the builder can download the resources required.

I kind of find this important to address and I am feeling down about the Google app store rejecting my breast icons all of a sudden, so you are in luck: I will try to fix this now!

[IzzySoft]

to get an API key from them so that the builder can download the resources required.

That will make reproducible builds (see Reproducible Builds, special client support and more at IzzyOnDroid – your app is listed there) impossible then, as builders wouldn't sign up for API keys. Maybe a better approach would be finding free replacements, e.g. at svgrepo.com (those are mostly licensed CC0 as far as I could see)?

so you are in luck: I will try to fix this now!

I didn't think I'd say "thank you, Google" one day and not meaning it as sarcasm 🤣 So I'll add a "Thank you, Paul" if you don't mind 😉

[MrApplejuice]

That will make reproducible builds (see Reproducible Builds, special client support and more at IzzyOnDroid – your app is listed there) impossible then, as builders wouldn't sign up for API keys.

That is very much true. When I created the app, I kind of realized that this was the only non-free aspect of it. On the other hand, I had to realize that creating my own icons or scraping open sources was very time intensive and just did not yield the same quality of results. Flaticon, even though it is non-free has a solid and appealing selection of icons and therefore saved me a lot of time making this app exist in the first place.

That said, for the builds, I am planning on creating a separate "source" for images based on a filler-texture. Then I can do full builds but I do not need to hook flaticon/freepik in the build system. This then opens up the possibility for alternative icon packs.

Alternative icon packs, however, have their own rats tail attached to them. svgrepo.com seems to be a pretty nice source, but requires CC attributions. Not a problem per-se, just a lot of work. So, getting alternative icon packs, or replacing the current one to get rid of the freepik-license would probably need to become its own issue.

There is also a completely different solution on the horizon that I discovered yesterday: The freepik api has changed once again, and it seems that the icon-downloads themselves are no longer hidden behind a api-key. This would be great: The images themselves are still protected by the freepik license itself (meh for an open source project, I know), but at the same time no API-key is needed for downloading them. I can actually investigate that solution before replacing the entire pictogram library.

[IzzySoft]

That said, for the builds, I am planning on creating a separate "source" for images based on a filler-texture. Then I can do full builds but I do not need to hook flaticon/freepik in the build system. This then opens up the possibility for alternative icon packs.

And that way, also opens a way to get rid of the NonFreeAssets anti-feature: the app itself could include a libre icon pack – which then can optionally replaced via manual download (on-demand from within the app, or simply by placing them into a specified/configurable location). Feeding three birds with one basket, so to say: no problematic build dependency, the AF dropped, and more flexibility/customization. I'd like that approach 😉

svgrepo.com seems to be a pretty nice source, but requires CC attributions.

I was just browsing it yesterday as we need some icons "refreshed" here. All the icons I took a closer look at were licensed CC0 (aka "public domain"). Why would those require attribution? Of course YMMV, as other icons might use other licenses – I didn't check all 500k of their collection.

freepik

That sounds like a possibility as well – if a "central attribution" (like "icons are ⓒ xxx by freepik") suffices. Though that might still require NonFreeAssets then. For dropping that AF, the "multiple icon packs" approach would be the better fit. But yeah, both would work.

[MrApplejuice]

Hey @IzzySoft - would you mind testing my branch #85 ? I updated the build-docs, added (working!) GitHub CI, and dropped the need of acquiring a freepik API key, but I am still relying on their resources.

However, I think that this should fully fix your issue?

[IzzySoft]

"Install Android SDK Platform 33 (revision 3)" finished.

> Task :app:checkKotlinGradlePluginConfigurationErrors
> Task :app:generateAboutStrings UP-TO-DATE
> Task :app:invokeHelpMakefile FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:invokeHelpMakefile'.
> A problem occurred starting process 'command 'make''

The process installed NDK v.26.1.10909125 "Side by side". Any dependency I miss? OK, make, and obviously imagemagick – but doing this during build is a bad idea:

convert resources/help_images/screenshots/help_diaper_logging.png -crop "+0+170" -crop "-0-135" app/src/main/res/drawable/help_diaper_logging.png

I doubt that is deterministic (converting images rarely is, so better use the "final ones" directly). And if it's not, it will make Reproducible Builds impossible (which is the reason I try to build the app myself, see Reproducible Builds, special client support and more at IzzyOnDroid).

Ah, and then python3. Looking at your github workflow (which I thought was for the desktop app) I guess the next thing it will fail to find is pandoc then… aw, no, first pipenv, python3-urllib3… Might be a good idea to list the build requirements somewhere 😉

So at this point, build crashes with

cd resources/help_images \
&& python3 -m pipenv install --skip-lock -r requirements.txt \
&& python3 -m pipenv run python process_markdown.py /build/repo/resources/help_images/help.md /build/repo/app/src/main/res/values/help_strings.xml
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3/dist-packages/pipenv/__main__.py", line 4, in <module>
    cli()
  File "/usr/lib/python3/dist-packages/pipenv/vendor/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/pipenv/vendor/click/core.py", line 696, in main
    with self.make_context(prog_name, args, **extra) as ctx:
  File "/usr/lib/python3/dist-packages/pipenv/vendor/click/core.py", line 621, in make_context
    self.parse_args(ctx, args)
  File "/usr/lib/python3/dist-packages/pipenv/vendor/click/core.py", line 1018, in parse_args
    rest = Command.parse_args(self, ctx, args)
  File "/usr/lib/python3/dist-packages/pipenv/vendor/click/core.py", line 875, in parse_args
    parser = self.make_parser(ctx)
  File "/usr/lib/python3/dist-packages/pipenv/vendor/click/core.py", line 821, in make_parser
    for param in self.get_params(ctx):
  File "/usr/lib/python3/dist-packages/pipenv/vendor/click/core.py", line 774, in get_params
    help_option = self.get_help_option(ctx)
  File "/usr/lib/python3/dist-packages/pipenv/cli.py", line 26, in get_help_option
    from .import core
  File "/usr/lib/python3/dist-packages/pipenv/core.py", line 21, in <module>
    import requests
  File "/usr/lib/python3/dist-packages/pipenv/vendor/requests/__init__.py", line 65, in <module>
    from . import utils
  File "/usr/lib/python3/dist-packages/pipenv/vendor/requests/utils.py", line 27, in <module>
    from .cookies import RequestsCookieJar, cookiejar_from_dict
  File "/usr/lib/python3/dist-packages/pipenv/vendor/requests/cookies.py", line 172, in <module>
    class RequestsCookieJar(cookielib.CookieJar, collections.MutableMapping):
AttributeError: module 'collections' has no attribute 'MutableMapping'
make: *** [help.makefile:23: app/src/main/res/values/help_strings.xml] Error 1
make: Leaving directory '/build/repo'

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:invokeHelpMakefile'.
> Process 'command 'make'' finished with non-zero exit value 2

And I haven't yet installed what I dug up from the requirement.txt files I found so far: panflute pandoc PyYAML (for the latter, I don't know which Debian package would be the matching one).

So simply running ./gradlew assembleRelease does not seem to do the job here, despite of having installed additional Debian packages:

extra_packages:
  - make
  - imagemagick
  - python3
  - pipenv
  - python3-urllib3

What build process would you advise then? In case it matters: the pod for building your app is using ubuntu:jammy here, with OpenJDK-21.

[MrApplejuice]

Hey! Sorry, I got very ill since yesterday.

One very quick pointer: https://github.com/babybuddy/babybuddy-for-android/blob/6588135e445aaf8fb3a4b516089b7030c43c1d83/README.md

I updated the readme with the required packages list, had to determine the list myself again. I am still running all of this on ubuntu 22.04.5, but annoyingly, some of the Android features rely on JDK 18, not 21, so a custom JDK is required.

Rest needs to wait until I am healthy again.

[IzzySoft]

some of the Android features rely on JDK 18, not 21

Oof. That will make it unfit for RB then. We only support long-term JDKs (11, 17, 21 currently – with 17 being the default for now). Features being dropped with later releases are one of the reasons to not use such interim-JDKs for "production".

Do you see any chance getting rid of that "unstable dependency", making the app build fine with 21 directly (i.e. not requiring JDK-18)?

[IzzySoft]

some of the Android features rely on JDK 18, not 21

which btw also tells you why you should not rely on such in-between versions but rather use LTS releases (currently 11, 17, 21) as reference for what features to use. Features introduced by those "experimental" JDKs might be dropped again once the next LTS is released…

[MrApplejuice]

Quick update from my side: I have recovered already and I did not forget about this. I tried to update to JDK 21, but I cannot get the builds to work. There seems to be "some weirdness" with how Android handles the JDK versions: https://developer.android.com/build/jdks . Took me some time to find this, I dug myself to that page via the gradle-docs, which was very convoluted.

As you correctly point out, Version 11 or 17 are the ones I should be going for. I hope that this will work, will test it over the next few days!

[IzzySoft]

Best luck! 🤞

Btw, JDK-11 is quite old. Please try 17 first if 21 doesn't work for you.

[MrApplejuice]

JDK 17 it is, #85 is merged. I think at least parts (huge parts) of your issues should have been addressed.

Please check the software list on https://github.com/babybuddy/babybuddy-for-android?tab=readme-ov-file#build-instructions, or alternatively the ci.yml file https://github.com/babybuddy/babybuddy-for-android/blob/master/.github/workflows/ci.yml. It looks quite promising that I was able to string things up so that they can be built here in GitHub now. I think this should mostly work.

I am not 100% sure if I can support 100% reproducible builds, I do not know the android toolchain well enough for that.

[IzzySoft]

Thanks, on it now. Seems your check script needs an update:

Check if python3, cython, and pipenv are installed...
cython is not installed

it's cython3 here (debian:bookworm as well as ubuntu:jammy). I see ubuntu:noble has a symlink for it; as that's what your build uses, I can switch to that instead. Getting past that point: your project is the first one (out of 370+) requiring me to explicitly set JAVA_HOME (as per check-build-prerequisites.sh) – and then fails with Android SDK for targetSdk 35 not found in /opt/sdk (which is AFAIK set up by gradle at a later point – that's why it's defined in app/build.gradle – same for NDK, which you did not define in build.gradle, nor anywhere else, so which NDK is needed?)

OK, so thanks to that check script I was able to complete most dependencies. Going by my assumption that gradle takes care for the rest, I now comment it out to see if the build works.

Checking the license for package NDK (Side by side) 27.0.12077973 in /opt/sdk/licenses
License for package NDK (Side by side) 27.0.12077973 accepted.
Preparing "Install NDK (Side by side) 27.0.12077973 v.27.0.12077973".

So yes, it does (also Android SDK for targetSdk 35) – but build fails:

convert-im6.q16: delegate failed `'rsvg-convert' -o '%o' '%i'' @ error/delegate.c/InvokeDelegate/1997.
convert-im6.q16: unable to open file `/tmp/magick-vK4ak8z-bAIU88W4lVpxPnmPvuelwgW2': No such file or directory @ error/constitute.c/ReadImage/620.
convert-im6.q16: no images defined `resources/tmp/_round_icon_cutout.png' @ error/convert.c/ConvertImageCommand/3234.
make: *** [resources.makefile:167: resources/tmp/_round_icon_cutout.png] Error 1

> Task :app:invokeResourcesMakefile FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:invokeResourcesMakefile'.
> Process 'command 'make'' finished with non-zero exit value 2

What might be missing there? Here's my recipe:

---
repository: https://github.com/babybuddy/babybuddy-for-android.git
updates: releases
notes:
  - using ubuntu:noble due to the cython3 version required
versions:
  - tag: v2.4.3
    apks:
      - apk_pattern: babybuddy-for-android-.*\.apk
        apk_url: https://github.com/babybuddy/babybuddy-for-android/releases/download/$$TAG$$/babybuddy-for-android-$$TAG$$.apk
        build:
          - export JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64
          - echo bash check-build-prerequisites.sh
          - ./gradlew assembleRelease -PskipIntegrationTests
          - find . -name '*.apk'
          - mv app/build/outputs/apk/release/*unsigned.apk /outputs/unsigned.apk
        build_home_dir: /build
        build_repo_dir: /build/repo
        build_user: build
        provisioning:
          android_home: /opt/sdk
          build_tools:
          cmake:
          cmdline_tools:
            version: '12.0'
            url: https://dl.google.com/android/repository/commandlinetools-linux-11076708_latest.zip
            sha256: 2d2d50857e4eb553af5a6dc3ad507a17adf43d115264b1afc116f95c92e5e258
          extra_packages:
            - python3
            - cython3
            - pipenv
            - pandoc
            - make
            - gcc
            - g++
            - curl
            - imagemagick
            - jq
          image: ubuntu:noble
          jdk: openjdk-17-jdk-headless
          ndk:
          platform:
          platform_tools:
          tools:
          verify_gradle_wrapper: true

Besides: it's a bad idea to convert images during build, that's almost sure to make RB fail (image conversion/compression usually is not deterministic), so better provide the "ready-to-use" images in the repo directly.