Skip to content

DAOS-18017 build: update mercury to 2.4.1 #16943

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
soumagne wants to merge 2 commits into
base: master
Choose a base branch
from
+1,609 −538
Open

DAOS-18017 build: update mercury to 2.4.1 #16943

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5d3a4b5
DAOS-18017 build: update mercury to 2.4.1
soumagne Oct 3, 2025
176d760
DAOS-18017 build: test new mercury-libfabric
soumagne Nov 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/bullseye-coverage.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -563,11 +563,13 @@ jobs:
STAGE_TAGS+=",provider"
if [[ '${{ matrix.stage }}' = *\ Verbs\ * ]]; then
FTEST_ARG+=' --provider ofi+verbs'
INST_RPMS+=' mercury-libfabric'
elif [[ '${{ matrix.stage }}' = *\ UCX\ * ]]; then
FTEST_ARG+=' --provider ucx+dc_x'
INST_RPMS+=' mercury-ucx'
elif [[ '${{ matrix.stage }}' = *\ TCP\ * ]]; then
FTEST_ARG+=' --provider ofi+tcp'
INST_RPMS+=' mercury-libfabric'
else
echo 'Unknown provider in ${{ matrix.stage }}'
exit 1
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/rpm-build-and-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -572,11 +572,13 @@ jobs:
STAGE_TAGS+=",provider"
if [[ '${{ matrix.stage }}' = *\ Verbs\ * ]]; then
FTEST_ARG+=' --provider ofi+verbs'
INST_RPMS+=' mercury-libfabric'
elif [[ '${{ matrix.stage }}' = *\ UCX\ * ]]; then
FTEST_ARG+=' --provider ucx+dc_x'
INST_RPMS+=' mercury-ucx'
elif [[ '${{ matrix.stage }}' = *\ TCP\ * ]]; then
FTEST_ARG+=' --provider ofi+tcp'
INST_RPMS+=' mercury-libfabric'
else
echo 'Unknown provider in ${{ matrix.stage }}'
exit 1
Expand Down
10 changes: 6 additions & 4 deletions Jenkinsfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
/* groovylint-disable ParameterName, VariableName */
/* Copyright 2019-2024 Intel Corporation
/* Copyright 2025 Google LLC
* Copyright 2025 Hewlett Packard Enterprise Development LP
* Copyright 2025-2026 Hewlett Packard Enterprise Development LP
* All rights reserved.
*
* This file is part of the DAOS Project. It is subject to the license terms
Expand All @@ -18,7 +18,7 @@

// To use a test branch (i.e. PR) until it lands to master
// I.e. for testing library changes
//@Library(value='pipeline-lib@your_branch') _
@Library(value='pipeline-lib@soumagne/mercury_libfabric') _
Copy link
Collaborator Author

@soumagne soumagne Nov 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@phender this is the best and easiest solution I think for now (using the PR daos-stack/pipeline-lib#480), this way there is only minimal changes required to the Jenkinsfiles.

Copy link
Contributor

@daltonbohning daltonbohning Nov 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reminder to ourselves that we need to land the pipeline-lib change first

Copy link
Collaborator Author

@soumagne soumagne Nov 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to be frank, I'm not completely sure of the order that should go in, if I look at the comment it says:

// The @library line needs to be edited in a PR for adding a test.
// Adding a test requires two PRs.  One to add the test.
// That PR should be landed with out deleting the PR branch.
// Then a second PR submitted to comment out the @Library line, and when it
// is landed, both PR branches can be deleted.

so in theory I think we want to land that PR first so that it uses the branch when it lands, then land the pipeline-lib PR without deleting the branch, and finally merge a new PR that comments out that line again so it uses the default branch. That's how I see it but maybe I'm missing something?

Copy link
Contributor

@daltonbohning daltonbohning Nov 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Usually we land the pipeline-lib PR (and delete the branch). Then update this daos PR back to the default / uncommented pipeline-lib. Then land this PR if testing is okay

Copy link
Collaborator Author

@soumagne soumagne Nov 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well imo if you do that then you are going to break all PRs / CIs until this PR has landed :)

Copy link
Contributor

@daltonbohning daltonbohning Nov 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I misunderstood what this PR was doing. I did some testing locally and for example trying to install this new mercury with 2.6.4 works as expected

sudo dnf install mercury-2.4.1~rc5-1.el8 daos-2.6.4
...
Installing:
 daos     2.6.4-7.3224.g8a20f44f.el8
 mercury  2.4.1~rc5-1.el8

So, PRs that do NOT have this change will be okay because their version of daos still depends on mercury and on libfabric.
Then, when pipeline-lib says to install mercury-libfabric it doesn't actually matter for those PRs because mercury-libfabric really just installs mercury and libfabric.

Copy link
Contributor

@daltonbohning daltonbohning Nov 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I see two options. Either what you mentioned about leaving the pipeline-lib reference in the landing of this PR, OR something like this:

  1. Remove the pipeline-lib reference in this PR.
  2. Keep the daos -> libfabric dependency for now for compatibility
  3. Land this PR, since with (1) and (2) it should still work.
  4. Land the pipeline-lib PR
  5. New daos PR to remove the daos -> libfabric dependency

IMO I prefer this approach so we don't have to leave the pipeline-lib reference in place

Copy link
Collaborator Author

@soumagne soumagne Nov 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PRs that do NOT have this change will be okay

you are allowed to install mercury without mercury-libfabric or mercury-ucx but all tests will fail because the providers / plugins won't be found. What this PR does is separate the libfabric mercury plugin that allows mercury to interface with libfabric. Without it, mercury cannot use libfabric.

Keep the daos -> libfabric dependency for now for compatibility

No, this does not help in any way and no with (1) and (2) it won't work because you do need mercury-libfabric. This is not a simple alias of some sort, to expand on what I just mentioned above, it installs a separate library that is similar to mercury-ucx (you can look at that as an example). This library is then dynamically loaded by mercury.

Copy link
Contributor

@daltonbohning daltonbohning Nov 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then I don't know how to land any of this without breaking CI

Copy link
Collaborator Author

@soumagne soumagne Nov 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well that's what I've just been trying to explain :) I'm glad we're reaching the same conclusion. The real issue is that rpms are globally visible once merged, so because of that, any change of that sort will face the same issues.


/* groovylint-disable-next-line CompileStatic */
job_status_internal = [:]
Expand Down Expand Up @@ -1048,7 +1048,8 @@ pipeline {
steps {
job_step_update(
testRpm(inst_repos: daosRepos(),
daos_pkg_version: daosPackagesVersion(next_version()))
daos_pkg_version: daosPackagesVersion(next_version()),
inst_rpms: 'mercury-libfabric')
)
}
post {
Expand Down Expand Up @@ -1100,7 +1101,8 @@ pipeline {
} */
job_step_update(
testRpm(inst_repos: daosRepos(),
daos_pkg_version: daosPackagesVersion(next_version()))
daos_pkg_version: daosPackagesVersion(next_version()),
inst_rpms: 'mercury-libfabric')
)
}
post {
Expand Down
4 changes: 2 additions & 2 deletions ci/provisioning/post_provision_config_common.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/bin/bash
#
# Copyright 2021-2023 Intel Corporation.
# Copyright 2025 Hewlett Packard Enterprise Development LP
# Copyright 2025-2026 Hewlett Packard Enterprise Development LP
#
# SPDX-License-Identifier: BSD-2-Clause-Patent
#
Expand Down Expand Up @@ -32,7 +32,7 @@ fi
# shellcheck disable=SC1091
. /etc/os-release
# shellcheck disable=SC2034
EXCLUDE_UPGRADE=mercury,daos,daos-\*
EXCLUDE_UPGRADE=mercury,mercury-\*,daos,daos-\*
if rpm -qa | grep mlnx; then
# packages not to allow upgrading if MLNX OFED is installed
EXCLUDE_UPGRADE+=,openmpi,\*mlnx\*,\*ucx\*
Expand Down
8 changes: 7 additions & 1 deletion ci/unit/required_packages.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
#!/bin/bash

#
# (C) Copyright 2025 Google LLC
# Copyright 2025-2026 Hewlett Packard Enterprise Development LP
#
# SPDX-License-Identifier: BSD-2-Clause-Patent
#
set -eu

# No longer used but provided by pipeline-lib
Expand All @@ -24,6 +29,7 @@ pkgs="$(utils/rpms/package_version.sh argobots lib) \
$(utils/rpms/package_version.sh libfabric debug) \
$(utils/rpms/package_version.sh mercury dev) \
$(utils/rpms/package_version.sh mercury debug) \
$(utils/rpms/package_version.sh mercury lib mercury_libfabric) \
$(utils/rpms/package_version.sh pmdk lib pmemobj) \
$(utils/rpms/package_version.sh pmdk debug pmemobj) \
$(utils/rpms/package_version.sh pmdk debug pmem) \
Expand Down
71 changes: 71 additions & 0 deletions deps/patches/mercury/0001_dep_versions.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
From c0d62973299b4ba98a9aaf9f82ed000a11c0f7b0 Mon Sep 17 00:00:00 2001
From: Jerome Soumagne <[email protected]>
Date: Mon, 24 Nov 2025 16:44:14 -0600
Subject: [PATCH] NA OFI/UCX: print version infos

NA OFI: fix log warning
---
src/na/na_ofi.c | 12 +++++++-----
src/na/na_ucx.c | 10 ++++++++++
2 files changed, 17 insertions(+), 5 deletions(-)

diff --git a/src/na/na_ofi.c b/src/na/na_ofi.c
index c7c3e0b3..682efe65 100644
--- a/src/na/na_ofi.c
+++ b/src/na/na_ofi.c
@@ -8048,8 +8048,10 @@ na_ofi_check_protocol(const char *protocol_name)
uint32_t runtime_version = fi_version();
na_return_t na_ret;

- NA_LOG_SUBSYS_DEBUG(cls, "Querying info on libfabric v%d.%d",
- FI_MAJOR(runtime_version), FI_MINOR(runtime_version));
+ NA_LOG_SUBSYS_INFO(cls,
+ "Querying info on libfabric (runtime v%d.%d, API v%d.%d)",
+ FI_MAJOR(runtime_version), FI_MINOR(runtime_version),
+ FI_MAJOR(FI_COMPILE_VERSION), FI_MINOR(FI_COMPILE_VERSION));
NA_CHECK_SUBSYS_ERROR(cls, FI_VERSION_LT(runtime_version, NA_OFI_VERSION),
out, accept, false,
"runtime libfabric version (v%d.%d) is lower than required version "
@@ -9105,7 +9107,7 @@ na_ofi_mem_handle_create(na_class_t NA_UNUSED *na_class, void *buf,

NA_LOG_SUBSYS_DEBUG(mem,
"Created mem handle %p (iov_base=%p, iov_len=%zu, iovcnt=1, "
- "flags=0x%lx, len=%zu)",
+ "flags=0x%" PRIx8 ", len=%" PRIu64 ")",
(void *) na_ofi_mem_handle, na_ofi_mem_handle->desc.iov.s[0].iov_base,
na_ofi_mem_handle->desc.iov.s[0].iov_len,
na_ofi_mem_handle->desc.info.flags, na_ofi_mem_handle->desc.info.len);
@@ -9444,8 +9446,8 @@ na_ofi_mem_handle_deserialize(na_class_t NA_UNUSED *na_class,
na_ofi_mem_handle->desc.info.iovcnt);

NA_LOG_SUBSYS_DEBUG(mem,
- "Deserialized mem handle %p (iov_base=%p, iov_len=%zu, iovcnt=%zu, "
- "flags=0x%lx, len=%zu)",
+ "Deserialized mem handle %p (iov_base=%p, iov_len=%zu, iovcnt=%" PRIu64
+ ", flags=0x%" PRIx8 ", len=%" PRIu64 ")",
(void *) na_ofi_mem_handle, na_ofi_mem_handle->desc.iov.s[0].iov_base,
na_ofi_mem_handle->desc.iov.s[0].iov_len,
na_ofi_mem_handle->desc.info.iovcnt, na_ofi_mem_handle->desc.info.flags,
diff --git a/src/na/na_ucx.c b/src/na/na_ucx.c
index 96501b27..7c0ac8d4 100644
--- a/src/na/na_ucx.c
+++ b/src/na/na_ucx.c
@@ -3433,8 +3433,18 @@ na_ucx_check_protocol(const char *protocol_name)
.field_mask = UCP_PARAM_FIELD_FEATURES, .features = NA_UCX_FEATURES};
ucp_context_h context = NULL;
ucs_status_t status;
+ unsigned int runtime_major_version, runtime_minor_version,
+ runtime_patch_version;
bool accept = false;

+ ucp_get_version(
+ &runtime_major_version, &runtime_minor_version, &runtime_patch_version);
+
+ NA_LOG_SUBSYS_INFO(cls,
+ "Querying info on UCX (runtime v%u.%u.%u, API v%u.%u)",
+ runtime_major_version, runtime_minor_version, runtime_patch_version,
+ UCP_API_MAJOR, UCP_API_MINOR);
+
status = ucp_config_read(NULL, NULL, &config);
NA_CHECK_SUBSYS_ERROR_NORET(cls, status != UCS_OK, done,
"ucp_config_read() failed (%s)", ucs_status_string(status));
110 changes: 0 additions & 110 deletions deps/patches/mercury/0001_na_ucx.patch
View file
Open in desktop

This file was deleted.

64 changes: 0 additions & 64 deletions deps/patches/mercury/0002_na_ucx_ep_flush.patch
View file
Open in desktop

This file was deleted.

Loading
Loading