unable to fork, possible reason: Cannot allocate memory

[wshands]

I'm seeing this error when running ArchR

Setting default genome to Hg38.
Using GeneAnnotation set by addArchRGenome(Hg38)!
Using GeneAnnotation set by addArchRGenome(Hg38)!
ArchR logging to : ArchRLogs/ArchR-createArrows-1a436e43eb13-Date-2021-07-22_Time-15-16-26.log
If there is an issue, please report to github with logFile!
2021-07-22 15:16:26 : Batch Execution w/ safelapply!, 0 mins elapsed.
2021-07-22 15:16:26 : (BAM_data : 1 of 1) Reading In Fragments from inputFiles (readMethod = bam), 0.001 mins elapsed.
2021-07-22 15:16:26 : (BAM_data : 1 of 1) Tabix Bam To Temporary File, 0.001 mins elapsed.
2021-07-22 15:18:10 : (BAM_data : 1 of 1) Reading BamFile 18 Percent, 1.736 mins elapsed.
2021-07-22 15:19:56 : (BAM_data : 1 of 1) Reading BamFile 35 Percent, 3.495 mins elapsed.
2021-07-22 15:21:04 : (BAM_data : 1 of 1) Reading BamFile 52 Percent, 4.637 mins elapsed.
2021-07-22 15:21:49 : (BAM_data : 1 of 1) Reading BamFile 70 Percent, 5.377 mins elapsed.
2021-07-22 15:22:40 : (BAM_data : 1 of 1) Reading BamFile 88 Percent, 6.228 mins elapsed.
2021-07-22 15:23:18 : (BAM_data : 1 of 1) Successful creation of Temporary File, 6.868 mins elapsed.
2021-07-22 15:23:18 : (BAM_data : 1 of 1) Creating ArrowFile From Temporary File, 6.868 mins elapsed.

************************************************************
2021-07-22 16:01:40 : ERROR Found in .tmpToArrow for (BAM_data : 1 of 1)
LogFile = ArchRLogs/ArchR-createArrows-1a436e43eb13-Date-2021-07-22_Time-15-16-26.log

<simpleError in mcfork(detached): unable to fork, possible reason: Cannot allocate memory>

************************************************************

2021-07-22 16:01:40 : createArrowFiles has encountered an error, checking if any ArrowFiles completed..
ArchR logging successful to : ArchRLogs/ArchR-createArrows-1a436e43eb13-Date-2021-07-22_Time-15-16-26.log
character(0)
Using GeneAnnotation set by addArchRGenome(Hg38)!
Using GeneAnnotation set by addArchRGenome(Hg38)!
Validating Arrows...
Getting SampleNames...

Copying ArrowFiles to Ouptut Directory! If you want to save disk space set copyArrows = FALSE

Getting Cell Metadata...
1 Error in h5checktypeOrOpenLoc(file, readonly = TRUE, fapl = NULL, native = native) :
  Error in h5checktypeOrOpenLoc(). Cannot open file. File '/mnt/ArchR/ArrowFiles/.arrow' does not exist.
Calls: ArchRProject ... FUN -> .getMetadata -> h5read -> h5checktypeOrOpenLoc
Execution halted

Any idea how to fix this? There seems to be plenty of memory:

Not much reported in log file:

...



************************************************************


2021-07-22 17:21:36 : createArrowFiles has encountered an error, checking if any ArrowFiles completed..

------- Completed

End Time : 2021-07-22 17:21:36
Elapsed Time Minutes = 44.3690342744191
Elapsed Time Hours = 0.73949182053407

[rcorces]

https://stackoverflow.com/questions/15668893/r-multicore-mcfork-unable-to-fork-cannot-allocate-memory

I dont think this is an ArchR issue. If I'm reading that linked post correctly, you may very well be running out of memory depending on how many threads you are using. Even though the log file doesnt say much to your eye, it is helpful if you upload it. But my interpretation is that you are indeed running out of memory.

[wshands]

Here's the log file
ArchR-createArrows-1b3a4608750-Date-2021-07-22_Time-16-37-13.log

[rcorces]

I would drop your threads, try 2 and see what happens

[wshands]

Thanks that fixed it!