merge refine-classification to latest version of Metabuli

Author: borijoa

.gitmodules

@@ -2,3 +2,6 @@
 	path = util/Metabuli-regression
 	url = https://github.com/steineggerlab/Metabuli-regression.git
 	branch = main
+[submodule "lib/mmseqs"]
+	path = lib/mmseqs
+	url = https://github.com/jaebeom-kim/MMseqs2.git

Dockerfile

@@ -1,8 +1,13 @@
 ARG APP=metabuli
-FROM --platform=$BUILDPLATFORM debian:stable as builder
+
+########################################
+# Builder stage (multi-arch cross-compile)
+########################################
+FROM --platform=$BUILDPLATFORM debian:stable AS builder
 ARG TARGETARCH
 ARG APP
 
+# Install build tools (including cross-compile libs)
 RUN dpkg --add-architecture $TARGETARCH \
     && apt-get update \
     && apt-get install -y \
@@ -12,8 +17,14 @@ RUN dpkg --add-architecture $TARGETARCH \
     && rm -rf /var/lib/apt/lists/*
 
 WORKDIR /opt/build
+
+# Copy in your repo (including .git and .gitmodules)
 ADD . .
 
+# Ensure submodules are initialized
+RUN git submodule update --init --recursive
+
+# Build three variants
 RUN if [ "$TARGETARCH" = "arm64" ]; then \
       mkdir -p build_$TARGETARCH/src; \
       cd /opt/build/build_$TARGETARCH; \

README.md

@@ -95,7 +95,7 @@ Metabuli also works on Linux ARM64 and Windows systems. Please check [https://mm
 
 ### Compile from source code
 ```
-git clone https://github.com/steineggerlab/Metabuli.git
+git clone --recurse-submodules https://github.com/steineggerlab/Metabuli.git
 cd Metabuli
 mkdir build && cd build
 cmake -DCMAKE_BUILD_TYPE=Release ..
@@ -178,34 +178,45 @@ This will generate three result files: `JobID_classifications.tsv`, `JobID_repor
 > Sankey diagram is available in the [GUI app](https://github.com/steineggerlab/Metabuli-App).
 
 #### JobID_classifications.tsv
-1. Classified or not
-2. Read ID
-3. Taxonomy identifier
-4. Effective read length
-5. DNA level identity score
-6. Classification Rank
-7. List of "taxID : k-mer match count"
+You can use `--lineage 1` option in `classify` module to print the full lineage next to `rank` column. 
+1. `is_classified`: Classified or not
+2. `name`: Read ID
+3. `taxID`: Tax. ID in the tax. dump files used in database creation
+4. `query_length`: Effective read length
+5. `score`: DNA level identity score
+6. `rank`: Taxonomic rank of the taxon
+7. `taxID:match_count`: List of "taxID : k-mer match count"
 
 ```
+#is_classified name  taxID query_length score rank taxID:match_count
 1 read_1  2688  294     0.627551 subspecies  2688:65
 1 read_2  2688  294     0.816327 subspecies  2688:78
 0 read_3  0     294     0        no rank
 ```
 
 #### JobID_report.tsv
-The proportion of reads that are assigned to each taxon.
+It follows Kraken2's report format. The first line is a header, and the rest of the lines are tab-separated values. The columns are as follows:
+
+1. `clade_proportion`: Percentage of reads classified to the clade rooted at this taxon
+2. `clade_count`: Number of reads classified to the clade rooted at this taxon
+3. `taxon_count`: Number of reads classified directly to this taxon
+4. `rank`: Taxonomic rank of the taxon
+5. `taxID`: Tax ID according to the taxonomy dump files used in the database creation
+6. `name`: Taxonomic name of the taxon
+
 ```
-33.73   77571   77571   0       no rank unclassified
-66.27   152429  132     1       no rank root
-64.05   147319  2021    8034    superkingdom      d__Bacteria
-22.22   51102   3       22784   phylum      p__Firmicutes
-22.07   50752   361     22785   class         c__Bacilli
-17.12   39382   57      123658  order           o__Bacillales
-15.81   36359   3       126766  family            f__Bacillaceae
-15.79   36312   26613   126767  genus               g__Bacillus
-2.47    5677    4115    170517  species               s__Bacillus amyloliquefaciens
-0.38    883     883     170531  subspecies                      RS_GCF_001705195.1
-0.16    360     360     170523  subspecies                      RS_GCF_003868675.1
+#clade_proportion  clade_count  taxon_count rank  taxID name  
+33.73   77571   77571   no rank 0        unclassified
+66.27   152429  132     no rank 1        root
+64.05   147319  2021    superkingdom  8034          d__Bacteria
+22.22   51102   3       phylum        22784         p__Firmicutes
+22.07   50752   361     class         22785            c__Bacilli
+17.12   39382   57      order         123658             o__Bacillales
+15.81   36359   3       family        126766              f__Bacillaceae
+15.79   36312   26613   genus         126767                 g__Bacillus
+2.47    5677    4115    species       170517                 s__Bacillus amyloliquefaciens
+0.38    883     883     subspecies    170531                        RS_GCF_001705195.1
+0.16    360     360     subspecies    170523                        RS_GCF_003868675.1
 
 ```
 
@@ -231,20 +242,19 @@ metabuli classifiedRefiner <i:read-by-read classification> <i:DBDIR> [options]
 * Options
    --threads : The number of threads to utilize (all by default)
    --remove-unclassified : Remove unclassified reads
-   --exclude-taxid : Remove list of taxids as well as its children
-   --select-taxid : Select list of taxids as well as its children
-   --select-columns : Select list of columns with number and handle full lineage as 7 (generated if absent)
+   --exclude-taxid : Remove list of taxids as well as its children (e.g., 1758,9685,1234)
+   --select-taxid : Select list of taxids as well as its children (e.g., 1758,9685,1234)
+   --select-columns : Select list of columns with number and handle full lineage as 7 (generated if absent) (e.g., 2,5,7,3)
    --report : Write report of refined classification file
    --rank : Adjust classification to the specified rank
-   --rank-file-type : Choose how to handle reads assigned to higher taxonomic ranks when using the --rank option. [0: without higher rank, 1: with higher rank, 2: separate file for higher rank classification]
+   --rank-file-type : Choose how to handle reads assigned to higher taxonomic ranks when using the --rank option. [0: exclude higher rank, 1: include higher rank, 2: make separate file for higher rank classification]
 
 ```
 #### Output
-- `JobID_refined.tsv`
+- refined classification file : `JobID_refined.tsv`
 - report : `JobID_refined_report.tsv`, `JobID_refined_krona.html`
 - higher rank classification file : `_refined_higherRanks.tsv`
 
-
 ---
 ## Extract 
 After running the `classify` command, you can extract reads that are classified under a specific taxon.

azure-pipelines.yml

@@ -89,6 +89,8 @@ jobs:
           ARCH: arm64
           CPREF: aarch64
     steps:
+      - checkout: self
+        submodules: true
       - script: |
           sudo dpkg --add-architecture $ARCH
           cat << HEREDOC | sudo tee /etc/apt/sources.list