Error at ATAC_Make_BigWig step: FT:Control.Workflow.Interpreter.Exec #46
Description
Hi,
I am facing an error when trying to run taiji on my RNAseq and ATACseq data. Everything looks fine until the pipeline reaches the first ATAC_Make_BigWig step, and then this error is returned for each sample before the pipeline exits:
$ taiji run --config config_bam.yml
[INFO][03-13 09:29] HiC_Read_Input(5cd2..): Running ...
[INFO][03-13 09:29] HiC_Read_Input(5cd2..): Complete!
[INFO][03-13 09:29] SCATAC_Read_Input(fa14..): Running ...
[INFO][03-13 09:29] SCATAC_Read_Input(fa14..): Complete!
[INFO][03-13 09:29] SCATAC_Make_Index(f732..): Running ...
[INFO][03-13 09:29] SCATAC_Make_Index(f732..): Complete!
[INFO][03-13 09:29] SCRNA_Read_Input(6cc6..): Running ...
[INFO][03-13 09:29] SCRNA_Read_Input(6cc6..): Complete!
[INFO][03-13 09:29] SCRNA_Make_Index(8dee..): Running ...
[INFO][03-13 09:29] SCRNA_Make_Index(8dee..): Complete!
[INFO][03-13 09:29] SCRNA_QC(1312..): Running ...
[INFO][03-13 09:29] SCRNA_QC(1312..): Complete!
[INFO][03-13 09:29] SCRNA_Merge_Matrix(1d67..): Running ...
[INFO][03-13 09:29] SCRNA_Merge_Matrix(1d67..): Complete!
[INFO][03-13 09:29] SCRNA_Merged_Reduce_Dimension(4520..): Running ...
[INFO][03-13 09:29] SCRNA_Merged_Reduce_Dimension(4520..): Complete!
[INFO][03-13 09:29] SCRNA_Merged_Batch_Correction(d5a2..): Running ...
[INFO][03-13 09:29] SCRNA_Merged_Batch_Correction(d5a2..): Complete!
[INFO][03-13 09:29] SCRNA_Merged_Make_KNN(41e8..): Running ...
[INFO][03-13 09:29] SCRNA_Merged_Make_KNN(41e8..): Complete!
[INFO][03-13 09:29] SCRNA_Merged_Get_Param(d3b5..): Running ...
[INFO][03-13 09:29] SCRNA_Merged_Get_Param(d3b5..): Complete!
[INFO][03-13 09:29] SCRNA_Merged_Cluster(4a69..): Running ...
[INFO][03-13 09:29] SCRNA_Merged_Cluster(4a69..): Complete!
[INFO][03-13 09:29] RNA_Read_Input(7785..): Running ...
[INFO][03-13 09:29] RNA_Read_Input(7785..): Complete!
[INFO][03-13 09:29] RNA_Download_Data(247f..): Running ...
[INFO][03-13 09:29] RNA_Download_Data(247f..): Complete!
[INFO][03-13 09:29] RNA_Download_Data(2ced..): Running ...
[INFO][03-13 09:29] RNA_Download_Data(2ced..): Complete!
[INFO][03-13 09:29] RNA_Download_Data(20f2..): Running ...
[INFO][03-13 09:29] RNA_Download_Data(20f2..): Complete!
[INFO][03-13 09:29] RNA_Download_Data(cbb6..): Running ...
[INFO][03-13 09:29] RNA_Download_Data(cbb6..): Complete!
[INFO][03-13 09:29] RNA_Download_Data(8f7d..): Running ...
[INFO][03-13 09:29] RNA_Download_Data(8f7d..): Complete!
[INFO][03-13 09:29] RNA_Make_Index(1ca6..): Running ...
[INFO][03-13 09:29] RNA_Make_Index(1ca6..): Complete!
[INFO][03-13 09:29] RNA_Convert_ID_To_Name(2391..): Running ...
[INFO][03-13 09:29] RNA_Convert_ID_To_Name(2391..): Complete!
[INFO][03-13 09:29] RNA_Make_Expr_Table(73a7..): Running ...
[INFO][03-13 09:29] RNA_Make_Expr_Table(73a7..): Complete!
[INFO][03-13 09:29] ATAC_Read_Input(4131..): Running ...
[INFO][03-13 09:29] ATAC_Read_Input(4131..): Complete!
[INFO][03-13 09:29] ATAC_Download_Data(ebc9..): Running ...
[INFO][03-13 09:29] ATAC_Download_Data(ebc9..): Complete!
[INFO][03-13 09:29] ATAC_Download_Data(d69f..): Running ...
[INFO][03-13 09:29] ATAC_Download_Data(d69f..): Complete!
[INFO][03-13 09:29] ATAC_Download_Data(b24d..): Running ...
[INFO][03-13 09:29] ATAC_Download_Data(b24d..): Complete!
[INFO][03-13 09:29] ATAC_Download_Data(bb62..): Running ...
[INFO][03-13 09:29] ATAC_Download_Data(bb62..): Complete!
[INFO][03-13 09:29] ATAC_Download_Data(6984..): Running ...
[INFO][03-13 09:29] ATAC_Download_Data(6984..): Complete!
[INFO][03-13 09:29] SCRNA_Make_Cluster_Matrix(7c56..): Running ...
[INFO][03-13 09:29] SCRNA_Make_Cluster_Matrix(7c56..): Complete!
[INFO][03-13 09:29] SCRNA_Make_Expr_Table(26ef..): Running ...
[INFO][03-13 09:29] SCRNA_Make_Expr_Table(26ef..): Complete!
[INFO][03-13 09:29] SCATAC_Get_Promoters(f0e7..): Running ...
[INFO][03-13 09:29] SCATAC_Get_Promoters(f0e7..): Complete!
[INFO][03-13 09:29] SCATAC_QC(7de3..): Running ...
[INFO][03-13 09:29] SCATAC_QC(7de3..): Complete!
[INFO][03-13 09:29] SCATAC_Pre_Get_Peak_List(b477..): Running ...
[INFO][03-13 09:29] SCATAC_Pre_Get_Peak_List(b477..): Complete!
[INFO][03-13 09:29] SCATAC_Merged_Feature_Selection(5cf7..): Running ...
[INFO][03-13 09:29] SCATAC_Merged_Feature_Selection(5cf7..): Complete!
[INFO][03-13 09:29] SCATAC_Merged_Spectral(e6a4..): Running ...
[INFO][03-13 09:29] SCATAC_Merged_Spectral(e6a4..): Complete!
[INFO][03-13 09:29] SCATAC_Merged_Reduce_Dims(dd94..): Running ...
[INFO][03-13 09:29] SCATAC_Merged_Reduce_Dims(dd94..): Complete!
[INFO][03-13 09:29] SCATAC_Merged_Batch_Correction(54ce..): Running ...
[INFO][03-13 09:29] SCATAC_Merged_Batch_Correction(54ce..): Complete!
[INFO][03-13 09:29] SCATAC_Merged_Make_KNN(987b..): Running ...
[INFO][03-13 09:29] SCATAC_Merged_Make_KNN(987b..): Complete!
[INFO][03-13 09:29] SCATAC_Merged_Cluster(e29a..): Running ...
[INFO][03-13 09:29] SCATAC_Merged_Cluster(e29a..): Complete!
[INFO][03-13 09:29] Confusion_Table(f7b8..): Running ...
[INFO][03-13 09:29] Confusion_Table(f7b8..): Complete!
[INFO][03-13 09:29] ATAC_Make_Index(71ac..): Running ...
[INFO][03-13 09:29] ATAC_Make_Index(71ac..): Complete!
[INFO][03-13 09:29] ATAC_Bam_To_Bed(97c9..): Running ...
[INFO][03-13 09:30] ATAC_Bam_To_Bed(97c9..): Complete!
[INFO][03-13 09:30] ATAC_Bam_To_Bed(0bbc..): Running ...
[INFO][03-13 09:30] ATAC_Bam_To_Bed(5972..): Running ...
[INFO][03-13 09:30] ATAC_Bam_To_Bed(0bbc..): Complete!
[INFO][03-13 09:30] ATAC_Bam_To_Bed(7a8a..): Running ...
[INFO][03-13 09:30] ATAC_Bam_To_Bed(5972..): Complete!
[INFO][03-13 09:31] ATAC_Bam_To_Bed(7a8a..): Complete!
[INFO][03-13 09:31] ATAC_Bam_To_Bed(ee00..): Running ...
[INFO][03-13 09:32] ATAC_Bam_To_Bed(ee00..): Complete!
[INFO][03-13 09:32] ATAC_Bam_To_Bed(fa4f..): Running ...
[INFO][03-13 09:32] ATAC_Bam_To_Bed(fa4f..): Complete!
[INFO][03-13 09:32] ATAC_Bam_To_Bed(909f..): Running ...
[INFO][03-13 09:32] ATAC_Bam_To_Bed(909f..): Complete!
[INFO][03-13 09:32] ATAC_Bam_To_Bed(67c9..): Running ...
[INFO][03-13 09:33] ATAC_Bam_To_Bed(a03f..): Running ...
[INFO][03-13 09:33] ATAC_Bam_To_Bed(67c9..): Complete!
[INFO][03-13 09:33] ATAC_Bam_To_Bed(d4ef..): Running ...
[INFO][03-13 09:33] ATAC_Bam_To_Bed(a03f..): Complete!
[INFO][03-13 09:34] ATAC_Bam_To_Bed(d4ef..): Complete!
[INFO][03-13 09:34] ATAC_Bam_To_Bed(b571..): Running ...
[INFO][03-13 09:34] ATAC_Bam_To_Bed(b571..): Complete!
[INFO][03-13 09:34] ATAC_Bam_To_Bed(1c79..): Running ...
[INFO][03-13 09:35] ATAC_Bam_To_Bed(d0be..): Running ...
[INFO][03-13 09:35] ATAC_Bam_To_Bed(1c79..): Complete!
[INFO][03-13 09:36] ATAC_Bam_To_Bed(d0be..): Complete!
[INFO][03-13 09:36] ATAC_Bam_To_Bed(714a..): Running ...
[INFO][03-13 09:37] ATAC_Bam_To_Bed(714a..): Complete!
[INFO][03-13 09:37] ATAC_Bam_To_Bed(a15c..): Running ...
[INFO][03-13 09:37] ATAC_Bam_To_Bed(a15c..): Complete!
[INFO][03-13 09:37] ATAC_Merge_Bed(fba0..): Running ...
[INFO][03-13 09:38] ATAC_Merge_Bed(fba0..): Complete!
[INFO][03-13 09:38] ATAC_Merge_Bed(211b..): Running ...
[INFO][03-13 09:40] ATAC_Merge_Bed(211b..): Complete!
[INFO][03-13 09:40] ATAC_Merge_Bed(9e1e..): Running ...
[INFO][03-13 09:42] ATAC_Merge_Bed(9e1e..): Complete!
[INFO][03-13 09:42] ATAC_Merge_Bed(aa11..): Running ...
[INFO][03-13 09:43] ATAC_Merge_Bed(aa11..): Complete!
[INFO][03-13 09:43] ATAC_Merge_Bed(cd5a..): Running ...
[INFO][03-13 09:44] ATAC_Merge_Bed(cd5a..): Complete!
[INFO][03-13 09:44] ATAC_Gene_Count(7e6c..): Running ...
[INFO][03-13 09:45] ATAC_Gene_Count(7e6c..): Complete!
[INFO][03-13 09:45] ATAC_Gene_Count(13e2..): Running ...
[INFO][03-13 09:45] ATAC_Gene_Count(13e2..): Complete!
[INFO][03-13 09:45] ATAC_Gene_Count(5eea..): Running ...
[INFO][03-13 09:45] ATAC_Gene_Count(5eea..): Complete!
[INFO][03-13 09:45] ATAC_Gene_Count(3850..): Running ...
[INFO][03-13 09:46] ATAC_Gene_Count(3850..): Complete!
[INFO][03-13 09:46] ATAC_Gene_Count(d376..): Running ...
[INFO][03-13 09:46] ATAC_Gene_Count(d376..): Complete!
[INFO][03-13 09:46] ATAC_Make_Expr_Table(4399..): Running ...
[INFO][03-13 09:46] ATAC_Make_Expr_Table(4399..): Complete!
[INFO][03-13 09:46] ATAC_Make_BigWig(d67c..): Running ...
needLargeMem: trying to allocate 0 bytes (limit: 100000000000)
[ERROR][03-13 09:47] ATAC_Make_BigWig(d67c..) Failed:
Ran commands:
bedGraphToBigWig ./tmp_dir_-60cc376ab093618b/tmp1 ./tmp_dir_-60cc376ab093618b/chr output_bam//ATACSeq/BigWig//CAF-PD106_ATAC_rep0.bw
which bedGraphToBigWig
Exception: error running: bedGraphToBigWig ./tmp_dir_-60cc376ab093618b/tmp1 ./tmp_dir_-60cc376ab093618b/chr output_bam//ATACSeq/BigWig//CAF-PD106_ATAC_rep0.bw
exit status: 255
stderr: needLargeMem: trying to allocate 0 bytes (limit: 100000000000)
CallStack (from HasCallStack):
error, called at src/Control/Workflow/Interpreter/Exec.hs:146:37 in SciFlow-0.8.0-IRKsT2ba9M716PeGlwt2FT:Control.Workflow.Interpreter.Exec
[INFO][03-13 09:47] ATAC_Make_BigWig(c39a..): Running ...
needLargeMem: trying to allocate 0 bytes (limit: 100000000000)
[ERROR][03-13 09:47] ATAC_Make_BigWig(c39a..) Failed:
Ran commands:
bedGraphToBigWig ./tmp_dir_-665fc58022b38ed6/tmp1 ./tmp_dir_-665fc58022b38ed6/chr output_bam//ATACSeq/BigWig//CAF-ctrl_ATAC_rep0.bw
which bedGraphToBigWig
Exception: error running: bedGraphToBigWig ./tmp_dir_-665fc58022b38ed6/tmp1 ./tmp_dir_-665fc58022b38ed6/chr output_bam//ATACSeq/BigWig//CAF-ctrl_ATAC_rep0.bw
exit status: 255
stderr: needLargeMem: trying to allocate 0 bytes (limit: 100000000000)
CallStack (from HasCallStack):
error, called at src/Control/Workflow/Interpreter/Exec.hs:146:37 in SciFlow-0.8.0-IRKsT2ba9M716PeGlwt2FT:Control.Workflow.Interpreter.Exec
[INFO][03-13 09:47] ATAC_Make_BigWig(dacb..): Running ...
needLargeMem: trying to allocate 0 bytes (limit: 100000000000)
[ERROR][03-13 09:47] ATAC_Make_BigWig(dacb..) Failed:
Ran commands:
bedGraphToBigWig ./tmp_dir_-6e7bd7fc064a03fa/tmp1 ./tmp_dir_-6e7bd7fc064a03fa/chr output_bam//ATACSeq/BigWig//NF1-ctrl_ATAC_rep0.bw
which bedGraphToBigWig
Exception: error running: bedGraphToBigWig ./tmp_dir_-6e7bd7fc064a03fa/tmp1 ./tmp_dir_-6e7bd7fc064a03fa/chr output_bam//ATACSeq/BigWig//NF1-ctrl_ATAC_rep0.bw
exit status: 255
stderr: needLargeMem: trying to allocate 0 bytes (limit: 100000000000)
CallStack (from HasCallStack):
error, called at src/Control/Workflow/Interpreter/Exec.hs:146:37 in SciFlow-0.8.0-IRKsT2ba9M716PeGlwt2FT:Control.Workflow.Interpreter.Exec
[INFO][03-13 09:47] ATAC_Make_BigWig(0b20..): Running ...
needLargeMem: trying to allocate 0 bytes (limit: 100000000000)
[ERROR][03-13 09:48] ATAC_Make_BigWig(0b20..) Failed:
Ran commands:
bedGraphToBigWig ./tmp_dir_-94927ee90f34d223/tmp1 ./tmp_dir_-94927ee90f34d223/chr output_bam//ATACSeq/BigWig//NF1-TGFb_ATAC_rep0.bw
which bedGraphToBigWig
Exception: error running: bedGraphToBigWig ./tmp_dir_-94927ee90f34d223/tmp1 ./tmp_dir_-94927ee90f34d223/chr output_bam//ATACSeq/BigWig//NF1-TGFb_ATAC_rep0.bw
exit status: 255
stderr: needLargeMem: trying to allocate 0 bytes (limit: 100000000000)
CallStack (from HasCallStack):
error, called at src/Control/Workflow/Interpreter/Exec.hs:146:37 in SciFlow-0.8.0-IRKsT2ba9M716PeGlwt2FT:Control.Workflow.Interpreter.Exec
[INFO][03-13 09:48] ATAC_Make_BigWig(f87d..): Running ...
needLargeMem: trying to allocate 0 bytes (limit: 100000000000)
[ERROR][03-13 09:48] ATAC_Make_BigWig(f87d..) Failed:
Ran commands:
bedGraphToBigWig ./tmp_dir_-fe947a6bedd3f4c9/tmp1 ./tmp_dir_-fe947a6bedd3f4c9/chr output_bam//ATACSeq/BigWig//NF1-TGFb-PD106_ATAC_rep0.bw
which bedGraphToBigWig
Exception: error running: bedGraphToBigWig ./tmp_dir_-fe947a6bedd3f4c9/tmp1 ./tmp_dir_-fe947a6bedd3f4c9/chr output_bam//ATACSeq/BigWig//NF1-TGFb-PD106_ATAC_rep0.bw
exit status: 255
stderr: needLargeMem: trying to allocate 0 bytes (limit: 100000000000)
CallStack (from HasCallStack):
error, called at src/Control/Workflow/Interpreter/Exec.hs:146:37 in SciFlow-0.8.0-IRKsT2ba9M716PeGlwt2FT:Control.Workflow.Interpreter.Exec
[ERROR][03-13 09:48] Program exits with errors
I have tried to see if it was a known problem of bedGraphToBigWig, and found this: https://groups.google.com/a/soe.ucsc.edu/g/genome/c/eU4eMBZRqKM
But I am not limiting the memory of my shell as they explain in that issue:
$ ulimit -a
real-time non-blocking time (microseconds, -R) unlimited
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 1029920
max locked memory (kbytes, -l) 32971788
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 1029920
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
I have also tried to use different installations of bedGraphToBigWig just in case it was a bad build, but the problem is the same installing it with conda and downloading it directly from UCSC.
I have also tried to run it both on a desktop PC and a server with 256GB RAM to see if that was the limiting factor, and the problem persists.
Any ideas what's going wrong with my analysis?
Thanks