Added nf-test by FernandoDuarteF · Pull Request #74 · UCLOrengoGroup/domain-annotation-pipeline

FernandoDuarteF · 2026-08-03T10:24:00Z

This PR has the same changes as #70, but it's not comparing to a forked repo this time.

I want to check whether CI works this time, as it was not working in the previous PR.

Removed 'dev' branch from push triggers in workflow.

chriswyatt1 · 2026-08-03T10:43:31Z

Just thought I would try removing dev from PR tests, but think this won't work. lets see (I will put back if not)

Updated results_dir to allow null value.

chriswyatt1 · 2026-08-03T14:56:11Z

@sillitoe @NSEdmunds . I think the PRs can be merged now if you are happy with them.

sillitoe · 2026-08-04T14:12:45Z

TODO:

add to CI
merge

sillitoe · 2026-08-05T10:10:10Z

@FernandoDuarteF is there any way of getting nf-test to be more specific about the differences in a failing test? I can't actually see where any differences are in this output.

sillitoe · 2026-08-05T10:10:59Z

Also, we merged the config PR #69 hence the conflict

FernandoDuarteF · 2026-08-05T10:20:42Z

It's mainly a problem of github, the output is truncated I believe.

The CI is failing here becuase of foldseek_parsed_results.tsv, it has slightly different results depending on the architecture. I got different results in some decimal places on my Mac, which is the one that I used for the snapshot.

I'll try to run the test on an amd machine and generate the snapshot again, that should work for the CI.

FernandoDuarteF · 2026-08-05T13:55:47Z

Hey everyone. It seems I always get a different snapshot on different platforms (ci vs codespaces vs my mac). It's starting to be a real headache.

Perhaps we shouldn't wait for this PR before running the pipeline on the next batch.

The final float in the following line is not stable when computed on different arch. ``` "A0A004_02\tf9ca035470c019ce20a74306af810176\tmed\t234-372_496-557\t201\t2\t29\t13\t5\t18\t11\t11.708\t0.319\t92.159\tproteome-tax_id-67581-0_v4\t\tStreptomyces viridosporus\tcellular organisms (no rank), Bacteria (domain), Bacillati (kingdom), Actinomycetota (phylum), Actinomycetes (class), Kitasatosporales (order), Streptomycetaceae (family), Streptomyces (genus)\t0.905\tTrue\t6gq3A02\t2.459e-12\t0.862\t3.40.50.620\tH\t0.995\t0.776\t81.270", ``` We have manaully changed `81.520` -> `81.270`, but this needs a better solution moving forwards.

sillitoe · 2026-08-05T15:39:52Z

Hey everyone. It seems I always get a different snapshot on different platforms (ci vs codespaces vs my mac). It's starting to be a real headache.

Perhaps we shouldn't wait for this PR before running the pipeline on the next batch.

Hmm.. interesting. The difference does seem more than expected just from calculations on different arch (81.520 -> 81.270).

I'm thinking it would be good to include this PR, so how about we:

manually change the single failing result in the snapshot (done)
add an issue to investigate this further

?

Also, just a note that I've added some changes to hopefully improve the reporting of failed nf-test results (unified diff + upload artifacts)

FernandoDuarteF · 2026-08-05T17:02:53Z

Yeah, maybe changing the snapshot manually is the only way. Or maybe we can add the changing files to the nf-test ignore?

I also noticed that the *chopped_pdbs.tar.gz files are have different md5s when I run the test in codespaces, which I found very strange as it doesn't happen in the ci or my mac. But I'll have a closer look.

sillitoe · 2026-08-05T17:12:00Z

Cool. Looks like tests are passing now. Are you happy to merge @FernandoDuarteF ?

FernandoDuarteF · 2026-08-05T17:15:48Z

Looks good! I'll merge then

sillitoe · 2026-08-05T17:15:57Z

Yeah, maybe changing the snapshot manually is the only way. Or maybe we can add the changing files to the nf-test ignore?

I also noticed that the *chopped_pdbs.tar.gz files are have different md5s when I run the test in codespaces, which I found very strange as it doesn't happen in the ci or my mac. But I'll have a closer look.

I think we'll definitely want to include the rest of the results file that has differences.

Very interesting to have an example chopped PDB file with changes - I would expect any changes would be a reflection on changes in the domain boundaries (in the results file). Strange to have differences in a single chopped PDB file without differences in the results file.

Happy to create new issues to look into both of these.

FernandoDuarteF · 2026-08-05T17:17:54Z

Yeah, maybe changing the snapshot manually is the only way. Or maybe we can add the changing files to the nf-test ignore?
I also noticed that the *chopped_pdbs.tar.gz files are have different md5s when I run the test in codespaces, which I found very strange as it doesn't happen in the ci or my mac. But I'll have a closer look.

I think we'll definitely want to include the rest of the results file that has differences.

Very interesting to have an example chopped PDB file with changes - I would expect any changes would be a reflection on changes in the domain boundaries (in the results file). Strange to have differences in a single chopped PDB file without differences in the results file.

Happy to create new issues to look into both of these.

I'll open the issues and explore a little bit.

FernandoDuarteF and others added 4 commits July 9, 2026 16:07

Added nf-test

0305758

Updated snapshot

4f8a2ea

Merge branch 'UCLOrengoGroup:main' into nf_test

4556936

Update workflow to trigger only on 'main' branch

0870339

Removed 'dev' branch from push triggers in workflow.

chriswyatt1 and others added 3 commits August 3, 2026 11:49

Add dev branch to build-test-push workflow

2479acb

Merge branch 'dev' into nf_test

ec3cfe2

Allow results_dir to be null in nextflow.config

ffb6560

Updated results_dir to allow null value.

FernandoDuarteF added 2 commits August 4, 2026 13:10

Updated snapshot with 6 new ids

5cb1f28

Updated README

245d106

FernandoDuarteF added 4 commits August 4, 2026 17:34

Added nf-test to CI

6b00e35

Minor fix

5328a29

Updated test file

fbedd9f

Minor fix to ci

54ea118

FernandoDuarteF and others added 7 commits August 5, 2026 11:24

Fixed round foloat not working properly

5f952f5

Added new snapshot

e6576e1

Merge branch 'dev' into nf_test

ac41f8a

Updated snapshot

98074bd

use unified diff to improve test differences

a858a05

on test failure upload full diff as artifact

6efaeb8

allow hidden files in artifact (.nf-test/)

3c71081

sillitoe added 3 commits August 5, 2026 16:21

be more specific about which nf-test artifacts to capture

5135173

ignore .nf-test

0b42c64

ignore nf-test binary

cb7149e

sillitoe added 2 commits August 5, 2026 16:42

HACK: manually adjust value due to minor rounding differences

3a2d17e

HACK: more minor manual changes to snapshot

7842929

sillitoe merged commit 2773ca2 into dev Aug 5, 2026
2 checks passed

Uh oh!

Conversation

FernandoDuarteF commented Aug 3, 2026

Uh oh!

chriswyatt1 commented Aug 3, 2026

Uh oh!

chriswyatt1 commented Aug 3, 2026

Uh oh!

sillitoe commented Aug 4, 2026

Uh oh!

sillitoe commented Aug 5, 2026

Uh oh!

sillitoe commented Aug 5, 2026

Uh oh!

FernandoDuarteF commented Aug 5, 2026

Uh oh!

FernandoDuarteF commented Aug 5, 2026

Uh oh!

sillitoe commented Aug 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

FernandoDuarteF commented Aug 5, 2026

Uh oh!

sillitoe commented Aug 5, 2026

Uh oh!

FernandoDuarteF commented Aug 5, 2026

Uh oh!

sillitoe commented Aug 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

FernandoDuarteF commented Aug 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

sillitoe commented Aug 5, 2026 •

edited

Loading

sillitoe commented Aug 5, 2026 •

edited

Loading