time to run 700K cells

[ahyeonhwang]

Hello,

It only took me an hour to run the ArchR brief tutorial for 100K cells, but it is taking me more than 12 hours to run 700K cells. Are there any tips and tricks to speed up this process?

Thank you very much!

[rcorces]

Please note that questions and feature requests belong in the Discussions forum. I have migrated this issue there. Issues are only for bug reports.

The only way to speed up computation is to increase the number of cores provided to ArchR via addArchRThreads(). We provide extensive benchmarking in the paper which should give you a good estimate for how much time it will take. Since you havent noted which steps you are running or how many cores / memory you have available, it is hard to provide further guidance.

[ahyeonhwang]

Thank you very much for your reply! When I request 20 cores, archr sets the default number of threads to 18. I also requested 100G of memory. The Louvain algorithm in the addClusters() function is where I am stuck.

Another question is regarding addHarmony(). I am getting this warning:
Warning messages:
1: Quick-TRANSfer stage steps exceeded maximum (= 39183900)
2: Quick-TRANSfer stage steps exceeded maximum (= 39183900)
3: Quick-TRANSfer stage steps exceeded maximum (= 39183900)
4: Quick-TRANSfer stage steps exceeded maximum (= 39183900)
5: Quick-TRANSfer stage steps exceeded maximum (= 39183900)
6: Quick-TRANSfer stage steps exceeded maximum (= 39183900)
7: Quick-TRANSfer stage steps exceeded maximum (= 39183900)
8: Quick-TRANSfer stage steps exceeded maximum (= 39183900)
9: Quick-TRANSfer stage steps exceeded maximum (= 39183900)
10: Quick-TRANSfer stage steps exceeded maximum (= 39183900)

Do you know what this warning means? How can I fix it?

Thank you very much!

[rcorces]

regarding addHarmony - https://stackoverflow.com/questions/21382681/kmeans-quick-transfer-stage-steps-exceeded-maximum
it looks like there isnt a way around this. ArchR allows you to pass whatever you want to Harmony but it doesnt look like Harmony allows you to change the arguments to kmeans.

regarding addClusters - this function is mostly a wrapper around Seurat::FindClusters(). If that is the slow step, then I think you have two options:

1. Tweak the parameters (for example sampleCells)
2. Look into how to speed that up on the Seurat side. I'm not sure what options are available but ArchR allows you to pass whatever you want to Seurat::FindClusters() via ...