Use of raw vs log-norm counts

[cv55]

Hello thanks for the great tool!

I ran sciRED twice using the raw and log-normalized counts and noticed that the results were highly similar.
Would it be wrong if I were to use to log-norm counts? I have to mention here that my data are coming from many difference 10X runs
so I expect some batch effect.

Also in our data we have 3' and 5' data. In your paper you mention that sciRED should be able to handle it quite well, but when I run a sample PCA on the top genes of the factor I noticed that the batch effect was still there. I was thinking of running sciRED twice, once for the 3' and then for the 5' data and then integrate the results of the factors that separate well my condition. (eg keeping only genes that have similar behaviour across the two modalities) . Do you thing this would be an option?

Thanks a lot.

[delipouya]

Hello - Thanks for your interest in sciRED.
Which step of sciRED are you applying to the log-normalized data?

The first step (GLM with a Poisson distribution) isn’t suitable for log-normalized counts because Poisson regression assumes a distribution over non-negative integer counts, whereas log-normalized data typically follows an approximately Gaussian distribution.

However, you can use log-normalized counts as input for the second step, where factors are identified. Essentially, this means replacing Pearson residuals with log-normalized data.

Handling batch effects is dataset-specific and depends on the strength and nature of the batch effect (linear vs. nonlinear). If you aim to identify shared gene programs across datasets or detect programs that vary between them, I recommend running sciRED on the merged dataset. If your experimental design allows, you can:

• Include batch as a covariate in the GLM step to reduce batch effects.
• Add batch as a covariate in the FCA table to check if factors are confounded by batch, then exclude those.

Running sciRED separately on each dataset is an option, but it reduces statistical power and might cause you to miss important biological signals. In complex datasets, we’ve found that splitting by lineage or cell type and running the pipeline separately for each subset can sometimes improve results.

Hope this helps! Let me know if you have any questions.

[cv55]

Thanks a lot for your quick reply.

What I mean by log-normalized counts is that in the scanpy object I replace the X matrix with the log-normalized counts, before running the proc.get_sub_data() to get the variable features. I am not sure if this matrix is used at any other point.

When not using the log-normalized counts one of the variable features I get in the end is MALAT1, which looks wrong, so that's why I went for the normalized counts in the X matrix.

In the step where the glm.poissonGLM(y, x) is applied, the x I am using is the library size per cell (as estimated from the raw counts). If I am not mistaken using non-integer values should return an error here.

Another question is that even though I setting the seed at the top of the script I always get slightly different results every time I run it. Is there a way to avoid it?

[delipouya]

Just make sure that the y input to glm.poissonGLM(y, x) consists of count data and does not contain decimal values.
Some functions in sciRED involve numerical optimization, which can lead to slight variations across runs. However, in our experience, the factors are generally robust to these hyperparameters, and such differences are negligible. You might also try setting a global seed to see if that helps:
https://stackoverflow.com/questions/11526975/set-random-seed-programwide-in-python