Only small fractions of cells are being retained after creation of Arrow Files + ArchRProject

[RK900]

Hello,

I'm trying to use ArchR for peak calling on this dataset, specifically these 4 (LLL_control, LLL_stim, DIG_control, DIG_stim).

My code (mostly taken directly from the tutorial):

library(ArchR)

addArchRThreads(threads = 1) 

# inputFiles <- getTutorialData("Hematopoiesis")
# inputFiles

inputFiles = c(
    "dogma/GSM5065524_LLL_CTRL_fragments.tsv.gz",
    "dogma/GSM5065527_LLL_STIM_fragments.tsv.gz",
    "dogma/GSM5065530_DIG_CTRL_fragments.tsv.gz",
    "dogma/GSM5065533_DIG_STIM_fragments.tsv.gz"
)
names(inputFiles) = c(
    "LLL_CTRL",
    "LLL_STIM",
    "DIG_CTRL",
    "DIG_STIM"
)

addArchRGenome("hg38")

ArrowFiles <- createArrowFiles(
  inputFiles = inputFiles,
  sampleNames = names(inputFiles),
  minTSS = 4, #Dont set this too high because you can always increase later
  minFrags = 1000, 
#   addTileMat = TRUE,
#   addGeneScoreMat = TRUE,
  subThreading = FALSE
)

proj1 <- ArchRProject(
  ArrowFiles = ArrowFiles, 
#   outputDirectory = "dogma_seq",
  copyArrows = TRUE #This is recommened so that you maintain an unaltered copy for later usage.
)

According to the corresponding filtered barcodes files, there should be thousands of cells, however, after I try

> proj1

I get only 38 cells

class: ArchRProject 
outputDirectory: /Users/rohankoodli/Documents/berkeleyresearch/benchmark_multiVI/ArchROutput 
samples(4): DIG_STIM DIG_CTRL LLL_CTRL LLL_STIM
sampleColData names(1): ArrowFiles
cellColData names(13): Sample TSSEnrichment ... nDiFrags BlacklistRatio
numberOfCells(1): 38
medianTSS(1): 4.257
medianFrags(1): 2595

Any pointers as to why this is happening?

[rcorces]

medianTSS(1): 4.257

Looks like your cells are barely passing your quality threshold of TSS > 4

[RK900]

Even if I set minTSS = 0 and minFrags = 0, I still only get 38 cells:

numberOfCells(1): 38
medianTSS(1): 4.257
medianFrags(1): 2595

[rcorces]

Sorry - something must be wrong on your end. I downloaded 2 of the dogma files and ran the following code and got the following project result. Maybe try using threads > 1

library(ArchR)

addArchRThreads(threads = 16) 

# inputFiles <- getTutorialData("Hematopoiesis")
# inputFiles
setwd("~/temp/dogma")

inputFiles = c(
  "GSM5065524_LLL_CTRL_fragments.tsv.gz",
  "GSM5065527_LLL_STIM_fragments.tsv.gz"
)
names(inputFiles) = c(
  "LLL_CTRL",
  "LLL_STIM"
)

addArchRGenome("hg38")

ArrowFiles <- createArrowFiles(
  inputFiles = inputFiles,
  sampleNames = names(inputFiles),
  minTSS = 4, #Dont set this too high because you can always increase later
  minFrags = 1000, 
  #   addTileMat = TRUE,
  #   addGeneScoreMat = TRUE,
  subThreading = FALSE, force = TRUE
)

proj1 <- ArchRProject(
  ArrowFiles = ArrowFiles, 
  outputDirectory = "test_first2",
  copyArrows = TRUE #This is recommened so that you maintain an unaltered copy for later usage.
)

> proj1

           ___      .______        ______  __    __  .______      
          /   \     |   _  \      /      ||  |  |  | |   _  \     
         /  ^  \    |  |_)  |    |  ,----'|  |__|  | |  |_)  |    
        /  /_\  \   |      /     |  |     |   __   | |      /     
       /  _____  \  |  |\  \\___ |  `----.|  |  |  | |  |\  \\___.
      /__/     \__\ | _| `._____| \______||__|  |__| | _| `._____|
    
class: ArchRProject 
outputDirectory: /corces/home/rcorces/temp/dogma/test_first2 
samples(2): LLL_CTRL LLL_STIM
sampleColData names(1): ArrowFiles
cellColData names(13): Sample TSSEnrichment ... nDiFrags BlacklistRatio
numberOfCells(1): 14749
medianTSS(1): 17.335
medianFrags(1): 8550

[RK900]

There must have been some issue when I created the Arrow files for the first time - forcing ArchR to overwrite the existing Arrow files with force=TRUE seems to have done the trick.