[FIX] Warning for discrete variable with >100 values in OWFile

[PrimozGodec]

Issue

Fixes #4094

Description of changes

As @janezd suggested the variable with more than 100 different values should not be DiscreteVariable. This PR:

• changes guessing mechanism such that the variable with more than 100 different values is not a StringVariable
• add tests for guess_data_type function
• OWFile warns the user who set the variable with more than 100 different values as a DiscreteVariable

Includes

• Code changes
• Tests
• Documentation

[janezd]

As you noted, there are still many ways to force variables to have >100 values. Having a warning in DiscreteVariable.__init__ doesn't do any good (nobody sees it), and raising an exception would be worse (nobody would catch it).

I wrote a comment in #4094 - maybe the file widget would just give a warning?

Automated detection should of course recognize such variables as strings. Wasn't it already?

[ajdapretnar]

Automated detection should of course recognize such variables as strings. Wasn't it already?

It did, I am sure of it.

I also remember some users asking to increase the number of allowed discrete variables, mostly biologists, geologists and physicists.

[PrimozGodec]

Automated detection should, of course, recognize such variables as strings. Wasn't it already?

Automated detection recognizes variables that have int(round(len(values)**.7)) < num_unique_variables as string where len(values) is number of items in the table. This threshold is dependent on the number of items size (which looks ok). Last time we had a case when we had about 30000 items and for this data table, this threshold is 1361. Orange recognized a variable that had a few hundred unique values as a discrete and the box-plot freeze. I know this case is very unusual but it can happen.

Should we put a threshold of 100 hundred unique values besides the current threshold for variable recognition or we leave it as it is?

I agree with the warning in the File widget.

[codecov]

Codecov Report

❗ No coverage uploaded for pull request base (master@1fb5f8a). Click here to learn what that means.
The diff coverage is 98.27%.

@@            Coverage Diff            @@
##             master    #4120   +/-   ##
=========================================
  Coverage          ?   85.68%           
=========================================
  Files             ?      390           
  Lines             ?    69726           
  Branches          ?        0           
=========================================
  Hits              ?    59743           
  Misses            ?     9983           
  Partials          ?        0

[PrimozGodec]

I removed the warning from the Discrete variable and added the Warning on the File widget. Since now there is a warning on the widget for discrete variables with more than 100 values I suggest that we keep the same rule for guessing (see the previous comment).

[janezd]

Domain editor is a component and as such it shouldn't expect the class into which it is inserted to have some specific warning that the component will trigger.

• I think the best solution would be to raise the warning from within OWFile widget. Currently it warns only if the user manually changes the type to categorical. The warning should also be shown in .tab files if the column is specified as discrete and contains too many values, not just if the user overrides the guessing heuristic.

• If we keep the current solution, I believe it would be better if domain editor itself declared the warning (we therefore treat it as a kind of "mixin"). You can do this by removing
  performance_warning = widget.Msg("Categorical variables with >100 values may decrease performance.") from widget's warnings and instead add

        widget.Warning.add_message(
            "performance_warning",
            "Categorical variables with >100 values may decrease performance.")

to domain editor's __init__.

I believe that the former is better, though.

[PrimozGodec]

I moved the warning triggering to the file widget and it shows the warning if the column in the file is marked as a discrete.