[Bug]: Logistic/Multinomial Regression classification table does not load

[vladimirsim]

JASP Version

0.19.1

Commit ID

84d54b934fa27731bb9eec44a4aa5f7ab0744dfd

JASP Module

Machine Learning

What analysis are you seeing the problem on?

Classification>Logistic/Multinomial

What OS are you seeing the problem on?

Windows 11

Bug Description

It is me again, sorry! :) I use the latest Nightly file from yesterday evening.
I am trying to train a Logistic/Multinomial Regression classification model but I get a 'factor xxx has new levels' error. I got this error for several different factors. What is very strange, is that when I add factors one by one, the problematic factor might load and the logistic regression classification will work, however when I load the next one or two variables (one by one or together), the error appears for a previously loaded factor.

Expected Behaviour

After the factor (variable) is loaded in the 'Features' compartment, the Logistic/Multinomial Regression classification table should load.

Steps to Reproduce

1. Train a model for logistic regression classification.
2. Once you start adding factors in the model, the error appears.

...

Log (if any)

Logs:
JASP 2024-11-28 17_03_34 Engine 0.zip

More Debug Information

Printscreen:

Final Checklist

• I have included a screenshot showcasing the issue, if possible.
• I have included a JASP file (zipped) or data file that causes the crash/bug, if applicable.
• I have accurately described the bug, and steps to reproduce it.

[koenderks]

Welcome back ;) It's good that you are testing this very thoroughly. Unfortunately I cannot reproduce this with another dataset... I did find this discussion https://stackoverflow.com/questions/22315394/factor-has-new-levels-error-for-variable-im-not-using on this issue, and I think it is caused by the test or training set having some variables with different factors. What if you fix the seed (default is random) in the advanced options so that you take the same training set every time, and then check if you can get it to work consistently.

[vladimirsim]

I tried to fix the seed but it didn't work. The same 'factor xxx has new level z' error appears. However, I discovered an interesting pattern. This error appears on probably 6 or 7 factors and for all of them except one the error appears for a level which has only one instance. For example, I get the 'factor KnowingCode has new level 7' and when I check the training set it turns out that only one loan has KnowingCode value of 7. Another example: 'factor Education has new level Master'. It turns out that only one loan has Education equal to Master in the training set. The same is valid for 4-5 other factors. There is only one factor which causes this error and which has more than one instance for each of its levels.

[koenderks]

Do you think you can make an example dataset with which I can reproduce this error? It is very difficult to debug this in my mind without a concrete example for which I can also test potential solutions.

[vladimirsim]

Ok, I will provide a dataset which will replicate the problem, but I would like to send it via email for safety reasons. Can you please provide an email address?