Add deacon and deacon DM #7473

SantaMcCloud · 2025-11-17T09:56:32Z

FOR CONTRIBUTOR:

I have read the CONTRIBUTING.md document and this tool is appropriate for the tools-iuc repo.
License permits unrestricted use (educational + commercial)
This PR adds a new tool or tool collection
This PR updates an existing tool or tool collection
This PR does something else (explain below)

SantaMcCloud · 2025-11-17T09:57:15Z

Test will error since there is still missing the format for deacon which has an open PR: galaxyproject/galaxy#21289

data_managers/data_manager_deacon/data_manager/deacon_datamanager.xml

bernt-matthias · 2025-11-18T09:00:02Z

data_managers/data_manager_deacon/data_manager/deacon_datamanager.xml

+            "path":"$link.strip('/')[-1]",
+        #end if
+        #if $input.is_select == "prebuild"
+            "dbkey":"@TOOL_VERSION@",


Why abuse dbkey to store tool version?

Do you know which genome versions have been used to generate these human / mouse genomes? Then we could set the dbkey correctly.

Seems to include GRCh38.p14 resp. GRCm39 but if I get it right there is other data included. So maybe the dbkey is not useful.

Why not also implement deacon index build to build indices for data from all_fasta

Why abuse dbkey to store tool version?

Do you know which genome versions have been used to generate these human / mouse genomes? Then we could set the dbkey correctly.

Seems to include GRCh38.p14 resp. GRCm39 but if I get it right there is other data included. So maybe the dbkey is not useful.

There are multiple source of data used for both prebuild index files so using them are not good in my eyes.

Why not also implement deacon index build to build indices for data from all_fasta

I can implement this aswell yes. I didnt thought about it since it only thought about using url to dowload any index file yet good idea!

Still: Why abuse dbkey to store tool version?

i did seen some tools did this so i thought i just use it aswell but i can change it for the prebuild files?

Storing the tool version is a really good idea. It just should be stored in a version column.

Either we use the dbkey column to store the information linking to the dbkeys provided by Galaxy, i.e. genome builds, or we remove the column (or leave it empty).

I will leave them empty then i think this is easier but if you prefer i could change them to something with informations

data_managers/data_manager_deacon/test-data/deacon.loc

tools/deacon/macros.xml

bernt-matthias · 2025-11-18T11:14:26Z

tools/deacon/macros.xml

+    </xml>
+    <xml name="deacon_build">
+        <param name="input" type="data" format="fastq,fastq.gz,fasta,fasta.gz" label="Input fasta/fastq file to build index file"/>
+        <param argument="-k" type="integer" value="31" label="Set kmer length" help="IMPORTANT: (kmer length + window size) - 1 has to be odd and smaller or equal to 96!"/>


again there so no min/max stated also they used a combination with k and w which has to be <= 96 and it has to be odd so therefore finding a good min/max is hard for it

I mean min=0 seems a good start :) ?

has to be odd

Can you add a validator for that?

I can try but the problem is that (k+w)-1 has to be odd not k itself. This means both w and k always has to be odd or even both. Im not sure if you can validate this in the validator tag but i could check it via commandline and return the run with no output but the msg that this doesnt function with the set parameter?

Then it's fine. A validator is not an option in this case, they only have access to the value itself.

So i can mark this as resolved? I also did add a min=1 to them

tools/deacon/deacon_index.xml

bernt-matthias · 2025-11-18T11:17:00Z

tools/deacon/macros.xml

+    </xml>
+    <xml name="deacon_union">
+        <repeat name="deacon_repeat" min="2" title="Input" default="2">
+            <conditional name="file_input">


Maybe just two input parameters with multiple="true", i.e. no repeat and no conditional

i can change it it yes but the idea behind this was that the user can input from his own data and can add the DM data to it/remove it or whatever function he used where i did use the repeat block. So to have this option a repeat block is needed atleast for the DM

Problem with repeats is that they are difficult to use in workflows since the number (and type?) of repeat units is fixed.

a repeat block is needed atleast for the DM

why? a select with multiple=true should work?

hm yes this will work did not thought about it like this sorry!

bernt-matthias · 2025-11-18T11:20:26Z

tools/deacon/macros.xml

+            </conditional>
+        </repeat>
+    </xml>
+    <xml name="deacon_diff">


Wondering if we need data from the data tables for these tools at all (diff, intersect, union). I see more use cases for user data.

I added the DM since in the future depending on how updated the index files from dev/other sourcec are that more information can be added to it since it is about host removal and when there are dataset which are implemented via DM are there they should be used so the user can build their own dataset based on the infomration they need. This is better then have like 1000 of deacon index build to transform all human genome data into a index file and then add other genome data to it which make over 1000 runs of combined deacon index runs if you get what i mean?

What would be the workflow? Did not get it yet

What i mean is this since when you are working with deacon you have to build the index files with all the information you want to filter out/not to filter out. This means you have to collect all the data in the begining, for example you want to build an index file for the human guts. This means you need to get all the data to build this index files which might take time. With a DM you can add this then and this can be used in a WF then without using the build index file always as an extra input. Also haveing the DM give the option to the user that in the futher new data can be add to the older index files which is less work for the user since he can use work from someone else to build his index file faster.

I hope this is a better explanation now! :)

OK. Lets keep diff as it is. Here, even the conditional is fine, since the order of the two files matters (I guess).

For the other (index) tools my suggestion would still be to remove conditionals if possible.

es the order does matter for this. I will remove then and make them simpler! for diff should i also change this so that there are no repeat blocks or are they fine?

SantaMcCloud · 2025-11-18T23:04:38Z

I will add the change later here since i have some testing to be done which i could done yet but some of reviews input was used to change the current state of the wrappers!

SantaMcCloud · 2025-11-19T20:10:12Z

@bernt-matthias i did change all and test all wrapper manually. When the format was added they all should be positiv!

bede · 2025-11-21T19:51:11Z

Deacon developer here. I'm unfamiliar with Galaxy development and how to test a PR like this in a running environment, but I'm happy to answer questions, review screenshots etc.

bede · 2025-11-21T19:53:52Z

data_managers/data_manager_deacon/data_manager/deacon_datamanager.xml

+        <conditional name="input">
+            <param name="is_select" type="select" label="Choose how to add data to the DM">
+                <option value="url">Copy an index file from an URL</option>
+                <option value="prebuild" selected="true">Download a pre-build file</option>


I'd suggest referring to "prebuilt" indexes throughout rather than "prebuild" or "pre-build"

I will change this then no worries!

bernt-matthias · 2025-11-21T20:05:12Z

Deacon developer here. I'm unfamiliar with Galaxy development and how to test a PR like this in a running environment, but I'm happy to answer questions, review screenshots etc.

I'm sure @SantaMcCloud can provide some screenshots of the tool if you like.

bernt-matthias · 2025-11-21T20:06:10Z

Btw. wonderful the get feedback from the developer of the underlying software. This is super helpful and much appreciated.

SantaMcCloud · 2025-11-21T21:34:53Z

Deacon developer here. I'm unfamiliar with Galaxy development and how to test a PR like this in a running environment, but I'm happy to answer questions, review screenshots etc.

The way a PR is tested is that the xml file has a test section where a test is written. This means the paramter which can be used are set therer together with how many outputs are generated and then this output can be check with different ways like size compare or if certain text is in the file for example. The Problem now is that in GitHub a Galaxy instance will be simulated and the each test will be run via commandline. Now the problem is that my test doesnt work is that Galaxy has datatype which are allowed to be used and i did use a datatype which is not yet implemented in Galaxy so Galaxy blocking the files which are set to this datatype to be used which will end up in a errored Test. I did test all test localy where you can add datatype without an problme and run the same simulation as GitHub does there all test where fine but when there is something where i have some troubles i will come back to this offer thank you! Also if you want too know more feel free to ask i can try to answer them or can provide any data/screenshots from implemente you tool to galaxy! :)

Btw. wonderful the get feedback from the developer of the underlying software. This is super helpful and much appreciated.

Yes this is really helpful thank you for this!

bernt-matthias · 2025-11-23T11:35:41Z

tools/deacon/deacon_index_diff.xml

+                <param name="k" value="31"/>
+                <param name="w" value="15"/> 


k+w is not odd here?

This calculation has to be odd: (k+w) -1. This means both k and w has to be odd or both has to be even otherwise the result of the calcualtion is not odd

bernt-matthias · 2025-11-23T11:38:14Z

tools/deacon/macros.xml

+            </conditional>
+        </repeat>
+    </xml>
+    <xml name="deacon_diff">


OK. Lets keep diff as it is. Here, even the conditional is fine, since the order of the two files matters (I guess).

For the other (index) tools my suggestion would still be to remove conditionals if possible.

bede · 2025-11-24T12:00:56Z

In case helpful, I've updated the usage section of the Deacon readme to cover set operations on indexes: https://github.com/bede/deacon/?tab=readme-ov-file#indexing.

SantaMcCloud added 3 commits November 17, 2025 10:47

Add deacon

130daa1

add .shed files

0e1bba1

typo

438cb98

SantaMcCloud mentioned this pull request Nov 17, 2025

Add idx format galaxyproject/galaxy#21289

Open

4 tasks

SantaMcCloud changed the title ~~Add deacon~~ Add deacon and deacon DM Nov 17, 2025

SantaMcCloud mentioned this pull request Nov 17, 2025

New tool request: Host decontamination tools #7438

Open

4 tasks

SantaMcCloud added 4 commits November 17, 2025 11:31

fix some tests

aae6216

change one test file

6e9014f

little changes

4e11579

little changes

8474b57

bernt-matthias reviewed Nov 18, 2025

View reviewed changes

SantaMcCloud added 9 commits November 18, 2025 23:10

typos

a7fc57f

add deacon build into dm

19ddd38

add dash to value for dm

f0ca31f

typo

18b9283

shorter desc in deacon_filter

8eff8c0

change unneeded double to single qoutes

57dfe9b

change fastq to sanger/illumina

142e8e3

change to paired collection input

42ee2f2

change test data format

6288d4e

split index into multiple wrappers

3c9a09d

bede reviewed Nov 21, 2025

View reviewed changes

bede mentioned this pull request Nov 21, 2025

Deposit panmouse-1 to Zenodo, all indexes going forwards bede/deacon#74

Closed

bernt-matthias reviewed Nov 23, 2025

View reviewed changes

review changes

3a31713

Add deacon and deacon DM #7473

Are you sure you want to change the base?

Add deacon and deacon DM #7473

Conversation

SantaMcCloud commented Nov 17, 2025

Uh oh!

SantaMcCloud commented Nov 17, 2025

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SantaMcCloud commented Nov 18, 2025

Uh oh!

SantaMcCloud commented Nov 19, 2025

Uh oh!

bede commented Nov 21, 2025

Uh oh!

bede Nov 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SantaMcCloud Nov 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bernt-matthias commented Nov 21, 2025

Uh oh!

bernt-matthias commented Nov 21, 2025

Uh oh!

SantaMcCloud commented Nov 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

bede Nov 21, 2025 •

edited

Loading

SantaMcCloud Nov 21, 2025 •

edited

Loading

SantaMcCloud commented Nov 21, 2025 •

edited

Loading