feature request: expand "Customizing Image Pull Behavior" to include a "platform" override

[alexanderankin]

In the "Customizing Image Pull Behavior" section:

https://java.testcontainers.org/features/configuration/#customizing-image-pull-behaviour

there is an ability to to specify the behavior of timeouts while pulling.

There is also "pull.policy" which is not documented here but only in another place (https://java.testcontainers.org/features/advanced_options/#image-pull-policy)

the idea is to add a "pull.platform" or "pull.default-platform" or something which will override the platform detected normally with the one given as a property.

The proposal, initially to provide a single value as an override, is amended to provide two settings. one of which is a list of platforms to attempt to use - say ["darwin/arm64", "linux/arm64", "linux/amd64"] - and another which is a boolean flag which says whether or not the first one is a strict requirement (if false - implicit last thing to try is the default behavior). if the flag is true, and the first parameter is not null, the image pull action fails if none of the archs are present.

and document it in this section.

pros:

satisfy the needs described here - #3810 (comment)

cons:

potentially a satisfactory solution can be contained in user code as here - https://java.testcontainers.org/features/advanced_options/#using-createcontainercmdmodifier

conclusion:

purpose of this issue is to determine whether or not this addition is welcome as it seems to solve a need based on the other thread (linked in pros section)

[ventod]

I would really appreciate having this feature implemented (or alternatively, support for forcing specific images by sha256 as discussed in #4762).

I'm developing a library that runs extensive tests against various DBMS. Some of these databases, rightfully, do not provide ARM-compatible images, and my development machine has an ARM CPU.

The inability to explicitly specify the platform (or exact image) leads to inconsistent testing. For example, a test might pass on a colleague's x86 machine but fail on mine due to differences between ARM and x86 images. This makes it difficult to ensure deterministic and reliable test behavior across environments.

Having a pull.platform or similar settings would greatly improve test reproducibility (and developer experience), especially in teams with different hardware configs.

[alexanderankin]

Ok so i have gotten some mixed reactions from team, but some words of approval mixed in there so I will prepare a PR and then advocate for it to be merged for this issue.

[eddumelendez]

Hi @alexanderankin, let's wait until the team reach out a decision about how to tackle this.

[alexanderankin]

Ok perfect, would be great to have final decision before any work gets done

[vellotis]

Hi! Any update on this?

[jbogard]

Just hit this on an M4 MBP trying to get the icr.io/db2_community/db2:12.1.0.0 image which does not support ARM.

[HofmeisterAn]

I have a quick question. If we set a default platform, wouldn't that always pull, for example, linux/amd64, even if a linux/arm64/v8 version exists? Is that really what we want?

OCI-compliant registries provide an API (/v2/<repo>/manifests/<tag>) to get the manifest, including all available architectures. Unfortunately, this API isn't accessible without authentication, so the biggest challenge is that we don't know which architectures exist before pulling.

Specifying the platform in the code doesn't make much sense either, since users run on different machines and container runtimes, including remote runtimes.

If a platform doesn't exist, the Docker Engine returns an error like:

no matching manifest for linux/arm64/v8 in the manifest list entries

Shouldn't we just fallback to pulling linux/amd64 (on macOS) and log a warning?

On top of that, we could allow a default platform configuration. Personally, I'd like to avoid running linux/amd64 if a linux/arm64/v8 version is available.

Maybe the Docker team could also help with an improved API to make this easier.

[alexanderankin]

i think for a testing tool a simple knob that works right is more important than never incurring the overhead, but I'm waiting to hear from the team to see if this change is even welcome - as of now my indication is that there is only a "null" answer, not a truthy one yet.

[ventod]

I have a quick question. If we set a default platform, wouldn't that always pull, for example, linux/amd64, even if a linux/arm64/v8 version exists? Is that really what we want?

OCI-compliant registries provide an API (/v2/<repo>/manifests/<tag>) to get the manifest, including all available architectures. Unfortunately, this API isn't accessible without authentication, so the biggest challenge is that we don't know which architectures exist before pulling.

Specifying the platform in the code doesn't make much sense either, since users run on different machines and container runtimes, including remote runtimes.

If a platform doesn't exist, the Docker Engine returns an error like:

no matching manifest for linux/arm64/v8 in the manifest list entries

Shouldn't we just fallback to pulling linux/amd64 (on macOS) and log a warning?

On top of that, we could allow a default platform configuration. Personally, I'd like to avoid running linux/amd64 if a linux/arm64/v8 version is available.

Maybe the Docker team could also help with an improved API to make this easier.

IMO if this issue is ever solved, I hope to have a per container knob that allows me to force a platform, "suggest" a list of allowed platform or use the default one (what the current machine has, aka current bheaviour).

[alexanderankin]

i edited my original post to reflect this suggestion. i think mode of "force", "list", "default" is redundant = force can just be a strict list with one element, list is the list (optionally strict), and default is no list. you only need two settings to implement this - a list and whether to strictly adhere to it.