Are CNMF & CNMF-E fully deterministic?

[kushalbakshi]

Hello,

I've been working with @MathiasMahn to set up and execute CaImAn from a DataJoint pipeline.

We're consistently noticing that even with the same parameters, on the same type of machine (HPC or high RAM EC2 instance), there is a slight difference in the output of the components in CNMF and CNMF-E when comparing results from the demo_pipeline.ipynb provided with the software, and when those steps are packaged into and executed within the DataJoint pipeline.

The differences range from 1-2 masks fewer to as many as 10-15 masks different depending on the dataset. Is this normal and expected behavior from the CNMF and CNMF-E algorithms to have small differences between runs or is this something where we should treat the demo_pipeline.ipynb results as the ground truth and investigate potential bugs in the DataJoint pipeline? Should we focus on the total number of components or is it better to focus on the "accepted" components?