Skip to content

Upgrade images to rendezvous hashing config#167

Merged
moroten merged 1 commit intobuildbarn:masterfrom
tyler-french:upgrade
May 30, 2025
Merged

Upgrade images to rendezvous hashing config#167
moroten merged 1 commit intobuildbarn:masterfrom
tyler-french:upgrade

Conversation

@tyler-french
Copy link
Contributor

@tyler-french tyler-french commented May 7, 2025

To use the configuration in: https://github.com/buildbarn/bb-adrs/blob/master/0011-rendezvous-hashing.md

Tested locally with a deployment

Pulling browser (ghcr.io/buildbarn/bb-browser:20250414t180739z-9b44996)...
20250414t180739z-9b44996: Pulling from buildbarn/bb-browser
688513194d7a: Already exists
bfb59b82a9b6: Already exists
efa9d1d5d3a2: Already exists
a62778643d56: Already exists
7c12895b777b: Already exists
3214acf345c0: Already exists
5664b15f108b: Already exists
0bab15eea81d: Already exists
4aa0ea1413d3: Already exists
da7816fa955e: Already exists
9aee425378d2: Already exists
f443df81d1df: Pull complete
Digest: sha256:ff8b742efbdce177a40beeec5e98c3e4536a686ee2d11a62cb3fb1fd6ae7c346
Status: Downloaded newer image for ghcr.io/buildbarn/bb-browser:20250414t180739z-9b44996
Starting docker-compose_scheduler_1                      ... done
Starting docker-compose_frontend_1                       ... done
Recreating docker-compose_browser_1                      ... done
Starting docker-compose_worker-hardlinking-ubuntu22-04_1 ... done
Starting docker-compose_storage-1_1                      ... done
Starting docker-compose_runner-installer_1               ... done
Starting docker-compose_storage-0_1                      ... done
Starting docker-compose_runner-hardlinking-ubuntu22-04_1 ... done
Attaching to docker-compose_scheduler_1, docker-compose_runner-installer_1, docker-compose_worker-hardlinking-ubuntu22-04_1, docker-compose_storage-0_1, docker-compose_runner-hardlinking-ubuntu22-04_1, docker-compose_storage-1_1, docker-compose_frontend_1, docker-compose_browser_1

@tyler-french tyler-french force-pushed the upgrade branch 2 times, most recently from f1cc079 to 0686f6b Compare May 7, 2025 16:06
Copy link
Collaborator

@moroten moroten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll merge it when all CI checks pass.

@tyler-french
Copy link
Contributor Author

I'll merge it when all CI checks pass.

Thanks, yep looking at what CI is checking, it seems to load the versions from the git_overrides

@HagegeR
Copy link

HagegeR commented May 28, 2025

I had issues with proxy and grpc in this new version, I hope someone here finds the way to overcome the issue, I believe it's related to this commit: buildbarn/bb-storage@214cfae

@moroten
Copy link
Collaborator

moroten commented May 28, 2025

Yes, we should upgrade the images. Just need CI to pass.

@tyler-french tyler-french force-pushed the upgrade branch 2 times, most recently from 99dcdae to c750e47 Compare May 28, 2025 18:13
@tyler-french
Copy link
Contributor Author

Just need CI to pass.

I don't seem to be able to start a CI run. Is there a way we can change the config to automatically start the CI on PR creation? That would reduce the iteration loop here

@moroten
Copy link
Collaborator

moroten commented May 28, 2025

Just need CI to pass.

I don't seem to be able to start a CI run. Is there a way we can change the config to automatically start the CI on PR creation? That would reduce the iteration loop here

I think GitHub will start them automatically once you've got your first PR merged. I'll look into it if that is not the case.

@moroten
Copy link
Collaborator

moroten commented May 28, 2025

All the configuration files are not in sync and the SHA1s in README.md need to be updated.

@tyler-french tyler-french force-pushed the upgrade branch 2 times, most recently from 370c804 to b3c6c6b Compare May 28, 2025 22:16
@moroten moroten changed the title chore(docker): upgrade images to new config Upgrade images to rendezvous hashing config May 29, 2025
@moroten
Copy link
Collaborator

moroten commented May 29, 2025

The patch files don't match any more. Compare with bb-storage and bb-remote-execution to see what needs to be changed.

@tyler-french
Copy link
Contributor Author

We should make some of these compatible with darwin, especially the scripts that use sed -i. It's tough to iterate when I can't reproduce things locally:

http_archive(
    name = "abseil-hello",
    patch_cmds = [
        "sed -i 's/com_google_absl/abseil-cpp/' BUILD.bazel",
        "sed -i 's/com_google_googletest/googletest/' BUILD.bazel",
    ],
    sha256 = "e676640e69e210636de795f571237bec09a9ad9af6e441bf56f0d193cfe1c9fc",
    strip_prefix = "abseil-hello-b4803b41ab3d58c503265148e5a7d3fd2a8e46d3/bazel-hello",
    urls = ["https://github.com/abseil/abseil-hello/archive/b4803b41ab3d58c503265148e5a7d3fd2a8e46d3.zip"],
)

For example:

➜  bb-deployments git:(upgrade) bazel fetch @abseil-hello
INFO: Repository +_repo_rules+abseil-hello instantiated at:
  <builtin>: in <toplevel>
Repository rule http_archive defined at:
  ...:233:21: An error occurred during the fetch of repository '+_repo_rules+abseil-hello':
   Traceback (most recent call last):
        File "...external/bazel_tools/tools/build_defs/repo/http.bzl", line 150, column 10, in _http_archive_impl
                patch(ctx)
        File "/../external/bazel_tools/tools/build_defs/repo/utils.bzl", line 233, column 21, in patch
                fail("Error applying patch command %s:\n%s%s" %
Error in fail: Error applying patch command sed -i 's/com_google_absl/abseil-cpp/' BUILD.bazel:
sed: 1: "BUILD.bazel": invalid command code B
ERROR: Error applying patch command sed -i 's/com_google_absl/abseil-cpp/' BUILD.bazel:
sed: 1: "BUILD.bazel": invalid command code B
INFO: Elapsed time: 0.262s
INFO: 0 processes.
ERROR: Build did NOT complete successfully
ERROR: Fetching some target dependencies failed with errors: Error applying patch command sed -i 's/com_google_absl/abseil-cpp/' BUILD.bazel:
sed: 1: "BUILD.bazel": invalid command code B
➜  bb-deployments git:(upgrade) 

@tyler-french
Copy link
Contributor Author

I created #168 as a follow-up

@moroten moroten merged commit dcfd162 into buildbarn:master May 30, 2025
1 check passed
@moroten
Copy link
Collaborator

moroten commented May 30, 2025

Thank you @tyler-french for sorting out this upgrade.

stagnation added a commit to stagnation/bb-deployments that referenced this pull request Aug 12, 2025
stagnation added a commit to stagnation/bb-deployments that referenced this pull request Aug 12, 2025
stagnation added a commit to stagnation/bb-deployments that referenced this pull request Aug 19, 2025
stagnation added a commit to stagnation/bb-deployments that referenced this pull request Aug 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants