[Bug]: Different LMM results on Mac vs PC (and vs older version)

[iandstephen]

JASP Version

0.19.1

Commit ID

No response

JASP Module

Mixed Models

What analysis are you seeing the problem on?

Linear Mixed Model

What OS are you seeing the problem on?

Windows 11, macOS Silicon

Bug Description

Running a Linear Mixed Model using one DV, one continuous Fixed Factor and one Random Factor. I am getting 3 different sets of results on v17.2 compared with v19.1 on a PC compared with v19.1 on a Mac. All options, data etc are identical.

v17.2 results:

v19.1 results on Mac:

v19.1 results on PC:

Data file as .csv
dragons.csv

Expected Behaviour

Produce all the same results.

Steps to Reproduce

...

Log (if any)

No response

More Debug Information

No response

Final Checklist

• I have included a screenshot showcasing the issue, if possible.
• I have included a JASP file (zipped) or data file that causes the crash/bug, if applicable.
• I have accurately described the bug, and steps to reproduce it.

[tomtomme]

@iandstephen
thx for the report. Satterthwaite is not described in the help file, but I guess it is based on some kind of bootstrap?
Setting the seed to a fixed number should give repeatable results

[FBartos]

Hi @iandstephen ,

Satterwaite is not based on a bootstrap so that should not be an issue.
It is a shame that you don't have a screenshot of the settings from 17.2 or the note from bellow the table --- the best available guess is that the random structure is not the same (as the overall F-test already differs, and one of the models clearly did not converge).

Frantisek

[iandstephen]

Hi @tomtomme I did look for bootstrap options but couldn't see any being used. These results are also repeatable within the version and OS - I was using it as a workshop exercise in a stats class and all the Macs were getting one result, all the lab PCs (which run the older version) were getting another result and all the laptop PCs (who had installed the newest version) were getting a third result. I have also tried setting different seeds in v18.3 and get the same result each time (I tried 1, 2 and 100).

@FBartos here is the setup and full results from the older version (v18.3 in this case but giving the same results as v17.2)

[FBartos]

@iandstephen , so, just to check that I understand well -- when you used the same settings in v.17 and v.18 , you verified you got the same results, right? So that should be correct.
And for the previous version you don't have the settings to verify, right?