Rework and centralize app image build and publish workflows for registries

[Rocket-Quack]

Hello everyone 🦆,

I am currently working on simplifying the operation of Docker for newbies or people who don't have much Docker experience with a script in the style of easy-install from Frappe using TUI (gum).
What I noticed during development and for some time now is that the workflows for building a Docker image differ depending on the repo and apparently also use different approaches.

I would like to tackle this and build the whole thing in a more systematic way.

I have already given it some thought, but so far I haven't come up with a solution that I think is great and should be used.

As a basis, I'm just going to take the four “big” Frappe apps (Erpnext, crm, helpdesk, and lms) and warn you that this could be a long text, but I think it's especially valuable for the future of Frappe apps and Docker.

First, I'd like to discuss the current situation.

Current Situation

The current setup appears to be historically split:
ERPNext triggers image builds centrally in frappe_docker, while CRM, Helpdesk and LMS still build in their own repositories (using frappe_docker layered build context) and publish mainly to GHCR.

This mixed model likely grew over time and is now harder to maintain consistently and is also difficult to understand from a technical perspective for Docker operation (except, of course, for people who build their own Docker image).

Proposed future

I propose a hybrid model for (now the big four/in the future all) the Frappe Apps.
Whats a Hybrid Model?
ERPNext, CRM, HRMS and LMS should only define when an image build is triggered and which app ref should be built, while frappe_docker should own the reusable workflow for how images are built, tested, tagged, and published. (ERPNext is doeing something like that already)

Practice example

In practice an app repo calls the reusable workflow in frappe_docker via workflow_call, passing inputs like app_repo, app_ref, and frappe_branch. The central workflow then builds the image using shared Dockerfiles from frappe_docker (typically images/layered/Containerfile), and pushes the resulting image to both registries (Docker Hub and GHCR app namespace, e.g. ghcr.io/frappe/crm).

flowchart TD
  A[frappe/crm release or dispatch] --> B[crm caller workflow]
  C[frappe/erpnext release or dispatch] --> D[erpnext caller workflow]
  B --> E[Checkout frappe_docker build context]
  D --> E
  E --> F[Build image]
  F --> H[(Docker Hub)]
  F --> I[(GHCR)]

Loading

Alternatives (maybe some bad ones and good ones)

I have also considered possible alternatives, but I am curious to hear your opinion

1. Keep the current setup
   Each app repository keeps its own build/publish workflow and only applies small fixes.
   Pros: no effort
   Cons: duplicated logic, inconsistent building/tagging/publishing, higher long-term maintenance

2. Fully centralized model
   Only frappe_docker triggers and builds all app images directly
   Pros: maximum central control and standardization
   Cons: stronger coupling, more operational bottleneck risk, less app-repo autonomy, repo developer lost authority over building

Additional topic: decouple ERPNext from default image building

images/production/Containerfile is also tightly coupled to ERPNext today (bench get-app erpnext is hardcoded), so the current default build path is not fully app free. (Maybe someone knows why and where this comes from i thin also historical 🤔 )
Because of this i suggest treating ERPNext decoupling as a dedicated follow-up step after workflow standardization.

This will likely require multiple PRs and should be done incrementally but in my opinion this is becoming increasingly necessary as the number of Frappe apps grows otherwise the current mixed setup will continue to increase overhead and complexity.

I think the more we pursue the approach of keeping everything related to Docker in Frappe Docker within reasonable limits, the better. Please feel free to share your thoughts or other approaches that I havent thought of yet.

[asieftejani]

LMS is a specific industry ... Helpdesk should be in the big 4

[Rocket-Quack]

LMS is a specific industry ... Helpdesk should be in the big 4

I haven't had much contact with LMS myself but sure. I simply sorted the top 4 projects (excluding ERPNext) in the Frappe Repo by stars and these were the ones that came up or are well known in the community. But of course this is not limited to these they are just mentioned as examples.

In principle, my idea applies to every official Frappe app (where it is worthwhile to build your own image due to the operating system).

[DanielRadlAMR]

I also think this repo should be the single source of truth for everything related to Frappe containers. Until now, I didn’t even know there are Frappe containers published only to GHCR and not to Docker Hub. This has bothered me throughout the framework: there often isn’t a clear path, and there are many individual solutions growing organically. If someone just wants to jump in, it’s hard to know where to start.

I think your suggested approach is the best; I don’t see any benefits in the alternatives.

However, this can’t be achieved by just changing this repo. Other app repos would need to implement changes as well. I’m not sure how easy it will be to get the maintainers of those repos to adopt our solution, as they might lose some control over their build processes. Perhaps we should raise an issue in the frappe repo to get a Frappe employee involved and encourage a streamlined container deployment process—otherwise, our effort might be less effective.

The ultimate goal should be for every official Frappe app to follow this process. As you said, this will likely be achieved gradually by migrating apps one by one. It makes sense to start with the biggest or most popular ones, but we are also dependent on when their maintainers implement or accept the changes, so it’s not entirely in our control.

I think the goal of your TUI should be to eventually replace the easy-install script. That script already depends heavily on our repo, and we can’t publish changes there ourselves. For example, it currently breaks due to our changes, and Bench maintainers haven’t merged fixing PRs like #1701.
Centralizing everything regarding deployment in this repo would make maintenance much easier than having it scattered across multiple repositories.

[asieftejani]

Hi @Rocket-Quack
Have you checked out https://github.com/rtCamp/Frappe-Manager
Its really amazing for beginers and you can add apps and do bench build
It might be close to what you ae trying to do

[Rocket-Quack]

I think the goal of your TUI should be to eventually replace the easy-install script. That script already depends heavily on our repo, and we can’t publish changes there ourselves. For example, it currently breaks due to our changes, and Bench maintainers haven’t merged fixing PRs like frappe/bench#1701.

In my current idea, I have also implemented TUI in such a way that these docker images are built themselves for this reason it is also possible to use your own apps or third-party apps outside of Frappe in a kind of future easy-docker environment.

However I think that the bench repo is rather the wrong place for this in the first place but it is now so.
I see more of a problem with the classification. From my point of view I would leave my TUI “helper” script in this repo, which not only contains further documentation but also enables the basic structure of the management of Frappe framework apps in Docker containers. At the end of the day, this also reduces overhead. Thats why I handled it explicitly even independently of the easy-install script. And as you say, I don't want to become dependent on another repo that has no influence on docker operation at the operating level, and vice versa.

[DanielRadlAMR]

Yeah, I meant to keep it in our repo as well. :)

I just wanted to point out that it should be capable of everything the easy-install script does, so it can fully replace it. By replacing, I mean removing it from the bench repo and maintaining it here instead.