200 - Bundle format and multiple secrets support (#198)

* Add bundle options
* Make script to setup and unpack secrets
* Add more guidance in README and bundle unbranded for testing
* Upload bundles
* Add .pepk generation
* Add more guides about how to publish.
* Remove `branded` target that cannot be used when having multiple signing keys and actually was never used
* Add signing config to the alerte_niger flavor
* Fix Alerte Niger icon

Author: mrsarm

.github/workflows/build.yml

@@ -45,9 +45,7 @@ jobs:
       env:
         ANDROID_SECRETS_KEY: ${{ secrets.ANDROID_SECRETS_KEY }}
         ANDROID_SECRETS_IV: ${{ secrets.ANDROID_SECRETS_IV }}
-      run: |
-        openssl aes-256-cbc -K $ANDROID_SECRETS_KEY -iv $ANDROID_SECRETS_IV -in secrets.tar.gz.enc -out ./secrets.tar.gz -d
-        tar -xf ./secrets.tar.gz
+      run: make org=medic keydec
 
     - name: Assemble unbranded
       if: ${{ env.ANDROID_SECRETS_KEY }}

.github/workflows/publish.yml

@@ -36,19 +36,24 @@ jobs:
     - name: Set up Fastlane
       run: gem install fastlane --no-document --quiet
 
-    - name: Unpack secrets
+    - name: Unpack secrets medic
       env:
         ANDROID_SECRETS_KEY: ${{ secrets.ANDROID_SECRETS_KEY }}
         ANDROID_SECRETS_IV: ${{ secrets.ANDROID_SECRETS_IV }}
-      run: |
-        openssl aes-256-cbc -K $ANDROID_SECRETS_KEY -iv $ANDROID_SECRETS_IV -in secrets.tar.gz.enc -out ./secrets.tar.gz -d
-        tar -xf ./secrets.tar.gz
+      run: make org=medic keydec
+
+    - name: Unpack secrets alerte_niger
+      env:
+        ANDROID_SECRETS_KEY: ${{ secrets.ANDROID_SECRETS_KEY_ALERTE_NIGER }}
+        ANDROID_SECRETS_IV: ${{ secrets.ANDROID_SECRETS_IV_ALERTE_NIGER }}
+      run: make org=alerte_niger keydec
 
     - name: Assemble unbranded
       uses: maierj/fastlane-action@v1.4.0
       with:
         lane: build
         options: '{ "flavor": "unbranded" }'
+
     - name: Assemble gamma
       uses: maierj/fastlane-action@v1.4.0
       with:
@@ -215,6 +220,19 @@ jobs:
       with:
         lane: build
         options: '{ "flavor": "alerte_niger" }'
+      env:
+        ANDROID_KEYSTORE_PATH: alerte_niger.keystore
+        ANDROID_KEYSTORE_PASSWORD: ${{ secrets.ANDROID_KEYSTORE_PASSWORD_ALERTE_NIGER }}
+        ANDROID_KEY_PASSWORD: ${{ secrets.ANDROID_KEY_PASSWORD_ALERTE_NIGER }}
+    - name: Bundle alerte_niger
+      uses: maierj/fastlane-action@v1.4.0
+      with:
+        lane: bundle
+        options: '{ "flavor": "alerte_niger" }'
+      env:
+        ANDROID_KEYSTORE_PATH: alerte_niger.keystore
+        ANDROID_KEYSTORE_PASSWORD: ${{ secrets.ANDROID_KEYSTORE_PASSWORD_ALERTE_NIGER }}
+        ANDROID_KEY_PASSWORD: ${{ secrets.ANDROID_KEY_PASSWORD_ALERTE_NIGER }}
 
     - name: GitHub release
       uses: softprops/action-gh-release@v1
@@ -226,5 +244,6 @@ jobs:
           build/outputs/apk/**/*-xwalk-arm64-v8a-release.apk
           build/outputs/apk/**/*-xwalk-armeabi-v7a-release.apk
           build/outputs/apk/**/*-webview-arm64-v8a-release.apk
+          build/outputs/bundle/**/*-release.aab
       env:
         GITHUB_TOKEN: ${{ secrets.RELEASE_TOKEN }}

.gitignore

@@ -8,14 +8,16 @@
 /.idea/
 *.iml
 
-secrets.tar.gz
-medic-official.keystore
+secrets*.tar.gz
+*.keystore
+*_private_key.pepk
 playstore-secret.json
+pepk.jar
 
 fastlane/*
 !fastlane/Fastfile
 
 .DS_Store
 .classpath
 .project
-.settings
+.settings

CHANGELOG.md

@@ -0,0 +1,93 @@
+# Release notes
+
+## 0.9.0
+
+### Changes
+
+- [feature] [cht-core#6741](https://github.com/medic/cht-core/issues/6741) Add more info about the version of the Android app :bookmark: cht-core v3.12.
+- [improvement] [#195](https://github.com/medic/cht-android/pull/195) New branding icon, and switch from "Medic Mobile" to "Medic".
+- [improvement] [#164](https://github.com/medic/cht-android/issues/164) Support Android 11.
+- [feature] [#206](https://github.com/medic/cht-android/issues/206) New branded app _Alerte Niger_.
+- [deprecation] Disable the _Demo_ app in the releases and in the Play Store (demo.dev.medicmobile.org is unmaintained).
+
+### Development changes
+
+- Upgrade Java and build dependencies. Now the base version is Java 11.
+- Support new Android App Bundle (`.aab`) required by the Play Store for new apps (for now only the new app _Alerte Niger_ uses it).
+- Add tooling to simplify the creation of new private keys for the Play Store.
+- Improvements in the linters configuration for better code quality.
+- Skip upload of APKs for `webview-armeabi-v7a` arch not used in releases.
+- Improvements in the Make config and more options available for developers using the CLI instead of the Android Studio.
+  Improve the development section in the README, and how to create new branded apps and private keys for signing.
+- Fix instrumentations tests (UI tests) to avoid random failures, and speed up executions in Github Actions using caches.
+- Fix Github Actions configuration to enable CI when the PR is created for an external contributor.
+
+## 0.8.0
+
+### Changes
+
+- [improvement] [#163](https://github.com/medic/cht-android/issues/163) New connection errors UX:
+  - The improvements only apply to _Webview_ flavors.
+  - It also applies when the app migrates to Webview from a XWalk installation.
+- [improvement] [#134](https://github.com/medic/cht-android/issues/134) New UX of Crosswalk to Webview migration:
+  - Add splash screen while the data is migrated.
+  - Fix bug that caused redirect to the login page after migrate.
+- [improvement] Remove unused `READ_EXTERNAL_STORAGE` from the `cmmb_kenya` and `surveillance_covid19_kenya` flavors
+
+You can see more about these changes in the release notes for [CHT Core v3.11.0](https://github.com/medic/cht-core/blob/master/release-notes/docs/3.11.0.md)
+
+## 0.7.3
+
+### Changes
+
+- [improvement] [cht-android#148](https://github.com/medic/cht-android/issues/148): Remove storage access for most flavors that don't use the permission
+- [improvement] New flavors added, and "livinggoods_innovation_ke" removed
+- [improvement] Add new translations for the prominent disclosure for location access: Tagalog (tel), Illonggo (hil), and Bisaya (ceb), and fixed translation string for the disclosure in Nepal (ne)
+
+### Notes
+
+A new flavor `unbranded_test` was added with the storage permission removed for testing, while `unbranded` flavor keeps the permission from the global manifest.
+
+## 0.7.0
+
+### Changes
+
+- [feature] [cht-android#136](https://github.com/medic/cht-android/issues/136): Add UI for prominent disclosure when requesting location permissions.
+
+### Notes
+
+The text used in the new location permission request is in the Android wrapper app itself (`cht-android`), and translated differently than [CHT Core labels](https://docs.communityhealthtoolkit.org/core/overview/translations/). Any additions or modifications to translations in `cht-android` are done in the `strings.xml` files according to the [Android localization framework](https://developer.android.com/guide/topics/resources/localization).
+
+
+## 0.6.0
+
+### Upgrade notes
+
+This release is largely intended to unblock the publishing of new apps onto the Google Play Store in a manner compatible with Android 10+. This release includes a difficult upgrade experience which may cause operational challenges in the field for apps with users on Android 10+. In particular:
+
+1. When loading the application after upgrade you will need an internet connection in order to cache the application code.
+2. The migration can take a few minutes to complete so after upgrade the user may be shown a login screen. If this happens, restart the application periodically until the user is logged in and the app loads as per usual.
+
+We are planning improvements to [make the migration more user friendly](https://github.com/medic/cht-android/issues/134) in a future release.
+
+Users on Android 9 and below do not need to be migrated and will be unaffected by this change. Because of this it is recommended that projects upgrade to v0.6.0 version of their app before issuing Android 10+ devices, if possible.
+
+Earlier releases are no longer accepted by the Google Play Store.
+
+### Changes
+
+- [improvement] [cht-android#106](https://github.com/medic/cht-android/issues/106): Update target SDK version to 29 for Play Store compliance.
+
+## 0.5.0
+
+### Upgrade notes
+
+This release changes the way in which location data is collected to better align with Play Store policies. Now the information is gathered only when filling in a form as opposed to as soon as the app is loaded.
+
+*Note*: This breaks backwards compatibility with older versions of the CHT Core Framework which may mean that location data is no longer collected at all. It is recommended you upgrade to CHT v3.9.2 or later before upgrading the android app.
+
+### Changes
+
+- [feature] [cht-core#6380](https://github.com/medic/cht-core/issues/6380): Adds intent so opening deployment URLs will prompt to load in app
+- [improvement] [cht-android#111](https://github.com/medic/cht-android/issues/111): Compliance with Play Store developer policies for PII collection disclosure
+- [bug] [cht-core#6648](https://github.com/medic/cht-core/issues/6648): Blank screen when launching external apps from CHT Android app

Makefile

@@ -3,6 +3,15 @@ GRADLE = ./gradlew
 GRADLE_OPTS = --daemon --parallel
 flavor = UnbrandedWebview
 abi = x86
+KEYTOOL = keytool
+OPENSSL = openssl
+JAVA = java
+BASE16 = base16
+RM_KEY_OPTS = -i
+APKSIGNER = apksigner
+
+# Public key from Google for signing .pepk files (is the same for all apps in the Play Store)
+GOOGLE_ENC_KEY = eb10fe8f7c7c9df715022017b00c6471f8ba8170b13049a11e6c09ffe3056a104a3bbe4ac5a955f4ba4fe93fc8cef27558a3eb9d2a529a2092761fb833b656cd48b9de6a
 
 ifdef ComSpec	 # Windows
   # Use `/` for all paths, except `.\`
@@ -11,8 +20,6 @@ ifdef ComSpec	 # Windows
 endif
 
 default: deploy logs
-branded: clean-apks assemble-all deploy-all logs
-branded-debug: clean-apks assemble-all-debug deploy-all logs
 xwalk: deploy-xwalk logs
 
 logs:
@@ -31,13 +38,26 @@ clean:
 clean-apks:
 	rm -rf build/outputs/apk/
 
-assemble:
-	${GRADLE} ${GRADLE_OPTS} assemble${flavor}
-assemble-all:
-	${GRADLE} ${GRADLE_OPTS} assembleRelease
+assemble: check-env
+	ANDROID_KEYSTORE_PATH=${ANDROID_KEYSTORE_PATH} ANDROID_KEY_ALIAS=${ANDROID_KEY_ALIAS} \
+	ANDROID_KEYSTORE_PASSWORD=${ANDROID_KEYSTORE_PASSWORD} ANDROID_KEY_PASSWORD=${ANDROID_KEY_PASSWORD} \
+	        ${GRADLE} ${GRADLE_OPTS} assemble${flavor}
+assemble-all: check-env
+	ANDROID_KEYSTORE_PATH=${ANDROID_KEYSTORE_PATH} ANDROID_KEY_ALIAS=${ANDROID_KEY_ALIAS} \
+	ANDROID_KEYSTORE_PASSWORD=${ANDROID_KEYSTORE_PASSWORD} ANDROID_KEY_PASSWORD=${ANDROID_KEY_PASSWORD} \
+	        ${GRADLE} ${GRADLE_OPTS} assembleRelease
 assemble-all-debug:
 	${GRADLE} ${GRADLE_OPTS} assembleDebug
 
+bundle: check-env
+	ANDROID_KEYSTORE_PATH=${ANDROID_KEYSTORE_PATH} ANDROID_KEY_ALIAS=${ANDROID_KEY_ALIAS} \
+	ANDROID_KEYSTORE_PASSWORD=${ANDROID_KEYSTORE_PASSWORD} ANDROID_KEY_PASSWORD=${ANDROID_KEY_PASSWORD} \
+	        ${GRADLE} ${GRADLE_OPTS} bundle${flavor}Release
+bundle-all: check-env
+	ANDROID_KEYSTORE_PATH=${ANDROID_KEYSTORE_PATH} ANDROID_KEY_ALIAS=${ANDROID_KEY_ALIAS} \
+	ANDROID_KEYSTORE_PASSWORD=${ANDROID_KEYSTORE_PASSWORD} ANDROID_KEY_PASSWORD=${ANDROID_KEY_PASSWORD} \
+	        ${GRADLE} ${GRADLE_OPTS} bundleRelease
+
 uninstall-all:
 	${GRADLE} uninstallAll
 
@@ -54,4 +74,139 @@ test: lint
 test-ui:
 	${GRADLE} connectedUnbrandedWebviewDebugAndroidTest -Pabi=${abi} --stacktrace
 test-ui-gamma:
-	${GRADLE} connectedMedicmobilegammaWebviewDebugAndroidTest -Pabi=${abi} --stacktrace
+	${GRADLE} connectedMedicmobilegammaWebviewDebugAndroidTest -Pabi=${abi} --
+
+
+#
+# "secrets" targets, to setup and unpack keystores
+#
+
+# Generate keystore
+keystore: check-org ${org}.keystore
+
+# Remove the keystore, the pepk file, and the compressed version
+keyrm: check-org
+	rm ${RM_KEY_OPTS} ${org}.keystore ${org}_private_key.pepk secrets/secrets-${org}.tar.gz
+
+# Remove all: the keystore, the encrypted version, the compressed version and the encrypted version
+keyrm-all: check-org
+	rm ${RM_KEY_OPTS} secrets/secrets-${org}.tar.gz.enc
+	${MAKE} keyrm
+
+# Remove the keystore and the compressed version, leaving only the encrypted version
+keyclean: check-org
+	rm ${RM_KEY_OPTS} ${org}.keystore secrets/secrets-${org}.tar.gz
+
+# Print info about the keystore
+keyprint: check-org check-env
+	${KEYTOOL} -list -v -storepass ${ANDROID_KEYSTORE_PASSWORD} -keystore ${org}.keystore
+
+# Print info about the key signature used in the apk release file
+keyprint-apk:
+	$(if $(shell which $(APKSIGNER)),,$(error "No command '$(APKSIGNER)' in $$PATH"))
+	$(eval APK := $(shell find build/outputs/apk -name \*-release.apk | head -n1))
+	${APKSIGNER} verify -v --print-certs ${APK}
+
+# Print info about the key signature used in the apk release file
+keyprint-bundle:
+	$(eval AAB := $(shell find build/outputs/bundle -name \*-release.aab | head -n1))
+	${KEYTOOL} -printcert -jarfile ${AAB}
+
+keygen: check-org keysetup secrets/secrets-${org}.tar.gz.enc
+
+keydec: check-org keysetup check-env
+	${OPENSSL} aes-256-cbc -iv ${ANDROID_SECRETS_IV} -K ${ANDROID_SECRETS_KEY} -in secrets/secrets-${org}.tar.gz.enc -out secrets/secrets-${org}.tar.gz -d
+	chmod go-rw secrets/secrets-${org}.tar.gz
+	${MAKE} keyunpack
+
+keyunpack: check-org
+	tar -xf secrets/secrets-${org}.tar.gz
+
+keysetup:
+	$(eval EXEC_CERT_REQUIRED = ${JAVA} ${KEYTOOL} ${OPENSSL})
+	$(info Verifing the following executables are in the $$PATH: ${EXEC_CERT_REQUIRED} ...)
+	$(foreach exec,$(EXEC_CERT_REQUIRED),\
+	        $(if $(shell which $(exec)),,$(error "No command '$(exec)' in $$PATH")))
+
+#
+# Intermediate targets for "secrets", don't use them
+#
+
+check-org:
+ifndef org
+	second_argument := $(word 2, $(MAKECMDGOALS) )
+	$(error "org" name not set. Try 'make org=name $(filter-out $@, $(MAKECMDGOALS))')
+endif
+ifeq ($(org),name)
+	$(error "org" cannot be equal to "name", it was just an example :S)
+endif
+ifdef org
+	$(eval ORG_UPPER := $(shell echo $(org) | tr [:lower:] [:upper:]))
+endif
+
+check-env:
+ifdef org
+	$(eval ORG_UPPER := $(shell echo $(org) | tr [:lower:] [:upper:]))
+endif
+ifndef ANDROID_SECRETS_IV
+	$(eval VARNAME=ANDROID_SECRETS_IV_${ORG_UPPER})
+	$(eval ANDROID_SECRETS_IV := $(shell echo ${${VARNAME}}))
+	$(eval VARNAME=ANDROID_SECRETS_KEY_${ORG_UPPER})
+	$(eval ANDROID_SECRETS_KEY := $(shell echo ${${VARNAME}}))
+	$(eval VARNAME=ANDROID_KEYSTORE_PASSWORD_${ORG_UPPER})
+	$(eval ANDROID_KEYSTORE_PASSWORD := $(shell echo ${${VARNAME}}))
+	$(eval VARNAME=ANDROID_KEY_PASSWORD_${ORG_UPPER})
+	$(eval ANDROID_KEY_PASSWORD := $(shell echo ${${VARNAME}}))
+	$(eval VARNAME=ANDROID_KEY_ALIAS_${ORG_UPPER})
+	$(eval ANDROID_KEY_ALIAS := $(shell echo ${${VARNAME}}))
+	$(eval VARNAME=ANDROID_KEYSTORE_PATH_${ORG_UPPER})
+	$(eval ANDROID_KEYSTORE_PATH := $(shell echo ${${VARNAME}}))
+endif
+
+${org}.keystore: check-org
+	$(if $(shell which $(BASE16)),,$(error "No command '$(BASE16)' in $$PATH"))
+	$(eval ANDROID_KEYSTORE_PASSWORD := $(shell ${BASE16} /dev/urandom | head -n 1 -c 16))
+	${KEYTOOL} -genkey -storepass ${ANDROID_KEYSTORE_PASSWORD} -v -keystore ${org}.keystore -alias medicmobile -keyalg RSA -keysize 2048 -validity 9125
+	chmod go-rw ${org}.keystore
+
+secrets/secrets-${org}.tar.gz: ${org}.keystore ${org}_private_key.pepk
+	tar -czf secrets/secrets-${org}.tar.gz ${org}.keystore ${org}_private_key.pepk
+	chmod go-rw secrets/secrets-${org}.tar.gz
+
+${org}_private_key.pepk: check-org pepk.jar
+	${JAVA} -jar pepk.jar --keystore=${org}.keystore --alias=medicmobile --keystore-pass=${ANDROID_KEYSTORE_PASSWORD} --output=${org}_private_key.pepk --include-cert --encryptionkey=${GOOGLE_ENC_KEY}
+	chmod go-rw ${org}_private_key.pepk
+
+pepk.jar:
+	$(info Downloading pepk.jar ...)
+	curl https://www.gstatic.com/play-apps-publisher-rapid/signing-tool/prod/pepk.jar -o pepk.jar
+
+secrets/secrets-${org}.tar.gz.enc: secrets/secrets-${org}.tar.gz
+	$(eval ANDROID_SECRETS_IV := $(shell ${BASE16} /dev/urandom | head -n 1 -c 32))
+	$(eval ANDROID_SECRETS_KEY := $(shell ${BASE16} /dev/urandom | head -n 1 -c 64))
+	$(eval ANDROID_KEYSTORE_PATH := $(org).keystore)
+	$(eval ANDROID_KEY_ALIAS := medicmobile)
+	${OPENSSL} aes-256-cbc -iv ${ANDROID_SECRETS_IV} -K ${ANDROID_SECRETS_KEY} -in secrets/secrets-${org}.tar.gz -out secrets/secrets-${org}.tar.gz.enc
+	chmod go-rw secrets/secrets-${org}.tar.gz.enc
+	$(info )
+	$(info ###########################################      Secrets!    ###########################################)
+	$(info #                                                                                                      #)
+	$(info # The following environment variables needs to be added to the CI environment                          #)
+	$(info # (Github Actions), and to your local environment if you also want                                     #)
+	$(info # to sign APK or AAB files locally:                                                                    #)
+	$(info #                                                                                                      #)
+	$(info )
+	$(info export ANDROID_KEYSTORE_PASSWORD_$(ORG_UPPER)=$(ANDROID_KEYSTORE_PASSWORD))
+	$(info export ANDROID_KEY_PASSWORD_$(ORG_UPPER)=$(ANDROID_KEYSTORE_PASSWORD))
+	$(info export ANDROID_SECRETS_IV_$(ORG_UPPER)=$(ANDROID_SECRETS_IV))
+	$(info export ANDROID_SECRETS_KEY_$(ORG_UPPER)=$(ANDROID_SECRETS_KEY))
+	$(info export ANDROID_KEYSTORE_PATH_$(ORG_UPPER)=$(ANDROID_KEYSTORE_PATH))
+	$(info export ANDROID_KEY_ALIAS_$(ORG_UPPER)=$(ANDROID_KEY_ALIAS))
+	$(info )
+	$(info #)
+	$(info # The file secrets/secrets-${org}.tar.gz.enc was created and has to be added to the git)
+	$(info # repository (don't worry, it's encrypted with some of the keys above).                                #)
+	$(info # NOTE: *keep the environment variables secret !!*                                                     #)
+	$(info #                                                                                                      #)
+	$(info ###########################################  End of Secrets  ###########################################)
+	$(info )