Prebuilt databases do not have nodes.dmp for the use of classifiedRefiner module; segfault if using gtdb_r226 instead of gtdb_r220+virus+human database

[TomKLHui]

(1) The prebuilt database worked well for the classify workflow but when I proceed to remove unclassified/ human portions it failed. should i just copy NCBI taxdump files here?
(2) Is it normal to have ~95% read as unclassified? What should I expect?

'''
classifiedRefiner 07a.metabuli/JL304_B27_2_classifications.tsv ../metabuli/gtdb+virus+human --threads 4 --remove-unclassified --report 1

Metabuli Version (commit): 1.1.1
Remove unclassified reads true
Exclude taxId as well as its children
Select taxId as well as its children
Select columns with number, (7:full lineage, generated if absent)
Make report of refined classification file true
Adjust classification to the specified rank
0: without higher rank, 1: with higher rank, 2: separate file for higher rank classification 0
Threads 4
Min. sequence similarity score 0

Loading nodes file ...File ../metabuli/gtdb+virus+human/nodes.dmp not found!
'''

(3) It also happened that the classify workflow only worked for trimmed reads using prebuilt gtdb_r220+virus+human but not for the prebuilt gtdb_r226. Any other use cases cause segfault. I wonder if it is caused by large data size limited RAM (remote server max: 500 Gb; input file size: ~5Gb , paired)

Tom

[jaebeom-kim]

Hi Thank you for reaching out!
(1) There is taxdump command to retreive taxdump files from a taxonomyDB file. Please try.
(2) Could you explain the sample and the database you used? and the version of Metabuli.
(3) As written in the update log, the new db format used in gtdb_r226 is not compatible to v1.1.1. Please find a binary fitting your machine here https://mmseqs.com/metabuli

I look forward hearing from you about (2).

[TomKLHui]

Thanks for the quick response.
(1) I did successfully run the refiner but it claimed that the refined_report exists even though it is not created anywhere. maybe it checked for the existance of input files instead?
'''
metabuli classifiedRefiner 07a.metabuli/sample_classifications.tsv ../gtdb_r226/metabuli/gtdb+virus+human --threads 4 --remove-unclassified --exclude-taxid 9606 --report 1
Metabuli Version (commit): 1.1.1
Remove unclassified reads true
Exclude taxId as well as its children 9606
Select taxId as well as its children
Select columns with number, (7:full lineage, generated if absent)
Make report of refined classification file true
Adjust classification to the specified rank
0: without higher rank, 1: with higher rank, 2: separate file for higher rank classification 0
Threads 4
Min. sequence similarity score 0

Loading nodes file ... Done, got 2726605 nodes
Loading merged file ... Done, added 96510 merged nodes.
Loading names file ... Done
Init computeSparseTable ...Done
Write refined classification result to:
07a.metabuli/sample_classifications_refined.tsv
07a.metabuli/sample_classifications_refined.tsv is already exists. <---NOT REALLY
Time for processing: 0h 0m 5s 627ms
'''

(2) The version of metabuli is v1.1.1 indeed, and the database is [gtdb+virus+human.tar.gz] downloaded from (https://metabuli.steineggerlab.workers.dev/gtdb+virus+human.tar.gz)
The sample was retrieved from deep-sea organism (with black corals / glass sponges), I haven't compared it to kraken2 output yet, but it might be possible that the databases didn't have extensive coverages on these environments esp. for the whole genome?
I have also used gtdb_r226 for the identification of the contigs and MAGs generated from these read and had decent coherence to the results here to genus level for common taxa and family/order level for rare taxa, if I just consider the classified portion of reads. However the classified portion is only 3.6% in this case

[jaebeom-kim]

Hi!

(1) I think I solved this issue in #161
Could you try with a binary from https://mmseqs.com/metabuli again?

(2) Could you share some reads if possible? I think random 10K reads would be enough. I will investigate if the low classified portion is error or not, and if there is a way to boost the portion. And it would be very helpful if you share Kraken2 output.

[TomKLHui]

(2) Here are the requested reads and reports
sample_report.tsv
sample_1.fq.gz
sample_2.fq.gz

The kraken2 output was based on the core_nt database
sample_kraken2_classification.txt
sample_kraken2_report.txt