Clarification on build dates for pre-compiled databases

[HenryCarr98]

Hi,

This is less of a development question and more me seeking clarity on the nr_euk database you feature. I wanted to check if the pre-compiled databases featured on here https://bioinformatics-centre.github.io/kaiju/downloads.html are the latest. most up to date? It would save a lot of overheads on my end if they were as recent as the ones produced by running the steps from here: https://github.com/bioinformatics-centre/kaiju?tab=readme-ov-file#creating-the-kaiju-index, i.e.,

mkdir kaijudb
cd kaijudb
kaiju-makedb -s <DB>

I would be most grateful if you could clarify this for me, Thanks.

[pmenzel]

Hi,

the databases available on the download page were built on the respective listed dates. Running kaiju-makedb would create a newer version.

[hyphaltip]

I have run on a fresh download of nr, taxonomy, prot.accession2taxid.FULL.gz and it only produces an empty kaiju_db_nr_euk.faa after processing. Is there any way to debug or understand what has failed?

pigz -dc nr.gz | kaiju-convertNR -m taxonomy/merged.dmp -t taxonomy/nodes.dmp -g prot.accession2taxid.gz -a \
-o kaiju_db_nr_euk.faa -e kaiju-excluded-accessions.txt -l kaiju-taxonlistEuk.tsv

#   Read the ~/README.md for more information about home directory contents.                       #
####################################################################################################

09:33:10 Reading taxonomic tree from file taxonomy/nodes.dmp
09:33:11 Reading file taxonomy/merged.dmp
09:33:11 Reading taxa from file kaiju-taxonlistEuk.tsv
09:33:11 Reading accession to taxon id map from file prot.accession2taxid.gz
09:44:37 Reading excluded accessions from file kaiju-excluded-accessions.txt
09:44:37 Processing NR file
12:26:26 Finished.

Result:

-rw-r--r-- 1 pkgadmin pkgadmin            1 Jan 29 12:26 kaiju_db_nr_euk.faa
-rw-r--r-- 1 pkgadmin pkgadmin        25110 Jan 28 16:32 kaiju-excluded-accessions.txt
-rw-r--r-- 1 pkgadmin pkgadmin          799 Jan 28 09:25 kaiju-taxonlistEuk.tsv
-rw-r--r-- 1 pkgadmin pkgadmin          248 Jan 28 16:33 make_kaiju.sh
-rw-r--r-- 1 pkgadmin pkgadmin 277715142600 Jan 28 22:08 nr.gz
-rw-r--r-- 1 pkgadmin pkgadmin  23393709125 Jan 23 21:26 prot.accession2taxid.gz
-rw-r--r-- 1 pkgadmin pkgadmin         1270 Jan 29 12:26 slurm-22394761.out
drwxr-xr-x 2 pkgadmin pkgadmin         4096 Jan 28 14:27 taxonomy

[hyphaltip]

one can dump out ncbi DB and set the taxonid in the name to support the Kaiju formatting expectation, I started a simple project to replace the convertNR with blastdbcmd so it can be generated from the downloads as FASTA downloads of nr are now no longer supported:
Attempting this here, not entirely done testing and fixing but maybe okay start - https://github.com/hyphaltip/kaiju_data_tools -

Current testing issues is creating FMI indexes on this NR file requires more than 1Tb of memory so am curious if nr_cluster will be suitable alternative db for the time being or not though recognize it represents collapsed proteins at 90% identity into a single representative (and thus taxon) so may not fulfill kaiju taxon recognition properly.