DAOS-17535 chk: misc improvements for CR logic #17329

Nasf-Fan · 2025-12-30T17:05:20Z

Include the followings:

When create CHK IV namespace, make the secondary group to be same as
the primary group. Otherwise, CHK logic may hit DER_NONEXIST trouble
when communicate via IV.
Integrate CHK IV namespace create and destroy API, cleanup related
logic, redefine the version.
Get ranks list and IV namespace version from CHK leader when rejoin.
Adjust CHK_REJOIN RPC for related changes.
Remove unsupported functionality for checking the specified 'phase'.
Add new test for case of lost some engine(s) before start checker.

Test-tag: recovery

Signed-off-by: Fan Yong [email protected]

Steps for the author:

Commit message follows the guidelines.
Appropriate Features or Test-tag pragmas were used.
Appropriate Functional Test Stages were run.
At least two positive code reviews including at least one code owner from each category referenced in the PR.
Testing is complete. If necessary, forced-landing label added and a reason added in a comment.

After all prior steps are complete:

Gatekeeper requested (daos-gatekeeper added as a reviewer).

github-actions · 2025-12-30T17:08:31Z

Ticket title is 'DAOS checker cannot completed on Aurora after some engines excluded'
Status is 'In Progress'
Labels: 'scrubbed_2.6.5'
https://daosio.atlassian.net/browse/DAOS-17535

daosbuild3 · 2025-12-30T17:57:45Z

Test stage NLT on EL 8.8 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17329/1/testReport/

daosbuild3 · 2025-12-31T03:18:18Z

Test stage Functional Hardware Medium MD on SSD completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17329/1/execution/node/1388/log

daosbuild3 · 2025-12-31T05:01:56Z

Test stage NLT on EL 8.8 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17329/2/testReport/

daosbuild3 · 2025-12-31T07:34:33Z

Test stage Functional on EL 8.8 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17329/2/execution/node/1076/log

daosbuild3 · 2026-01-01T04:35:55Z

Test stage NLT on EL 8.8 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17329/5/testReport/

daosbuild3 · 2026-01-01T08:42:03Z

Test stage Functional Hardware Medium MD on SSD completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17329/5/execution/node/1324/log

Include the followings: 1. When create CHK IV namespace, make the secondary group to be same as the primary group. Otherwise, CHK logic may hit DER_NONEXIST trouble when communicate via IV. 2. Integrate CHK IV namespace create and destroy API, cleanup related logic, redefine the version. 3. Get ranks list and IV namespace version from CHK leader when rejoin. Adjust CHK_REJOIN RPC for related changes. 4. Remove unsupported functionality for checking the specified 'phase'. 5. Add new test for case of lost some engine(s) before start checker. Test-tag: recovery Signed-off-by: Fan Yong <[email protected]>

daosbuild3 · 2026-01-04T03:37:40Z

Test stage NLT on EL 8.8 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17329/6/testReport/

Nasf-Fan · 2026-01-04T09:34:14Z

Test stage NLT on EL 8.8 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17329/6/testReport/

NLT failure for DAOS-17435, not related with the patch. All required CR tests passed.

Nasf-Fan · 2026-01-09T03:13:27Z

Ping reviewers, thanks!

kjacque

Overall looks good, just a couple minor comments/questions.

kjacque · 2026-01-10T00:35:53Z

src/chk/chk_engine.c

+	if (unlikely(ins->ci_ranks == NULL))
+		D_GOTO(out, rc = -DER_NOTAPPLICABLE);


This seems like a strange case where it would be good to log something. What could cause the ranks not to be fetched, if an error wasn't returned?

This may happen when chk_prop_fetch() got empty rank_list if checker has never run before or related CR property was removed for some reason. Anyway, it is not fatal for current operation since we expect to remove the specified rank from the rank_list.

kjacque · 2026-01-10T00:40:46Z

src/chk/chk_internal.h

 	uint32_t		 ci_start_flags;
+	uint32_t                 ci_ns_ver;


minor - indent is strange here. If the formatter did it, feel free to disregard.

Yes, it is the clang-format required, honestly, I also do not like such style;)

kjacque · 2026-01-10T00:57:09Z

src/chk/chk_iv.c

+	/* Let secondary rank == primary rank. */
+	rc = crt_group_secondary_modify(ins->ci_iv_group, ins->ci_ranks, ins->ci_ranks,
+					CRT_GROUP_MOD_OP_REPLACE, ns_ver);


Since this function call is repeated in a few places with the same comment, you could put it in a simple (maybe even inline) wrapper function that takes only ins and ns_ver as params. Also, I think it would be good to add detail to the comment as to why the primary and secondary groups must be the same for the checker.

Good idea. I will do that when need to refresh the patch next time, or in subsequent CR related PR.

Nasf-Fan changed the title ~~DAOS-17535 chk: secondary group should be same as primary group for C…~~ DAOS-17535 chk: misc improvements for CR logic Dec 31, 2025

Nasf-Fan force-pushed the Nasf-Fan/DAOS-17535_7 branch from 8e4ad6a to 639a8ec Compare December 31, 2025 03:16

Nasf-Fan force-pushed the Nasf-Fan/DAOS-17535_7 branch from 639a8ec to 78579dd Compare December 31, 2025 07:33

Nasf-Fan force-pushed the Nasf-Fan/DAOS-17535_7 branch 2 times, most recently from aa39da7 to 476d0f9 Compare January 1, 2026 03:09

Nasf-Fan force-pushed the Nasf-Fan/DAOS-17535_7 branch from 476d0f9 to 09aaf91 Compare January 4, 2026 02:41

Nasf-Fan marked this pull request as ready for review January 4, 2026 09:34

Nasf-Fan requested review from gnailzenh, kjacque and wangshilong and removed request for kjacque January 4, 2026 09:34

kjacque reviewed Jan 10, 2026

View reviewed changes

		if (unlikely(ins->ci_ranks == NULL))
		D_GOTO(out, rc = -DER_NOTAPPLICABLE);

DAOS-17535 chk: misc improvements for CR logic #17329

Are you sure you want to change the base?

DAOS-17535 chk: misc improvements for CR logic #17329

Uh oh!

Conversation

Nasf-Fan commented Dec 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Steps for the author:

After all prior steps are complete:

Uh oh!

github-actions bot commented Dec 30, 2025

Uh oh!

daosbuild3 commented Dec 30, 2025

Uh oh!

daosbuild3 commented Dec 31, 2025

Uh oh!

daosbuild3 commented Dec 31, 2025

Uh oh!

daosbuild3 commented Dec 31, 2025

Uh oh!

daosbuild3 commented Jan 1, 2026

Uh oh!

daosbuild3 commented Jan 1, 2026

Uh oh!

daosbuild3 commented Jan 4, 2026

Uh oh!

Nasf-Fan commented Jan 4, 2026

Uh oh!

Nasf-Fan commented Jan 9, 2026

Uh oh!

kjacque left a comment

Choose a reason for hiding this comment

Uh oh!

kjacque Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Nasf-Fan Jan 11, 2026

Choose a reason for hiding this comment

Uh oh!

kjacque Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Nasf-Fan Jan 11, 2026

Choose a reason for hiding this comment

Uh oh!

kjacque Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Nasf-Fan Jan 11, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

4 participants

Nasf-Fan commented Dec 30, 2025 •

edited

Loading