[ENH] File: Allow selecting files with arbitrary extensions

[janezd]

Issue

Resolves #6792. Requires biolab/orange-widget-base#278 (without, it should work, but without offering the . option).

Description of changes

Following the discussion with @markotoplak, I propose the following.

Readers are determined as follows:

1. If the file is taken from recent paths, it contains the format setting.
2. Otherwise, the type is determined by the extension.
3. If the extension is not known, the type is read in the combo (which the user must set).

Notes:

• For any loaded file, if the user changes the selection in the combo, this is store in recent paths, so rule 1 applies.
• For any file that is loaded anew, the first rule no longer applies.
• For successive files with unknown extensions, the rule 3 applies without resetting the combo.

Usage scenarios

Here are a few interesting sequences and the way they are resolved according to the above rules, which should also serve to write tests if we agree upon this.

In the following "Load" refers to clicking ... and selecting the file in directory, and "Select" refers to selecting the file from the recent file list.

1. The widget stores reader settings.

• Select Iris.tab. Combo is auto detect, file is read.
• Load a-tab-file.xlsx (a tab file, but with wrong extension). Combo is auto detect, a reading error appears.
• In combo, select Tab. File is read.
• Select Iris. Combo goes to auto detect.
• Reselect a-tab-file.xlsx. Combo goes to Tab, file is read.
• Alt 1:
  • Select Iris. Combo goes to auto detect.
  • Load a-tab-file.xlsx: the file is not taken from recent paths, so combo is auto detect, a reading error appears.
• Alt 2:
  • Load a-tab-file.xlsx: the file is not taken from recent paths, so combo is auto detect, a reading error appears.
• Alt 3:
  • Reload. Probably take the format from recent paths.

2. Reading subsequent files keeps the format.

• Select Iris.tab. Combo is auto detect, file is read.
• Load a-tab-file.foo. Combo is auto detect, a reading error appears.
• Set the correct type in combo: Tab. File is read.
• Load b-tab-file.foo. Combo remains Tab, file is read.

(... continues in 3)

3. Known extensions have precedence over the combo (for files that are loaded, not selected from recent paths).

(continued from above)

• Alt 1: Load b-tab-file.xlsx. Combo is auto detect, an read error appears because the widget attempts to read this as Excel.
• Alt 2: Load any Excel file. Combo is auto detect, it is read as Excel.
• Alt 3: Load b-tab-file.xlsx. Combo is auto detect, it is read as Excel.

Notes

• Apart from add_all="*" and a few different error messages, the most complicated change is in _get_reader. Reviewer should concentrate on this.
• mark_problematic_reader was needed because the combo state was not changed to the reader from the recent path. This is no longer the case.
• _initialize_reader_combo could be shorter (block signals, reinitialize, set the current index to previous value), but the current form makes its function more explicit.

Includes

• Code changes
• Tests
• Documentation

[codecov]

Codecov Report

Attention: Patch coverage is 94.73684% with 2 lines in your changes missing coverage. Please review.

Project coverage is 88.73%. Comparing base (b101640) to head (7a4537f).
Report is 9 commits behind head on master.

❌ Your patch check has failed because the patch coverage (94.73%) is below the target coverage (95.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6894      +/-   ##
==========================================
+ Coverage   88.71%   88.73%   +0.01%     
==========================================
  Files         332      332              
  Lines       73388    73409      +21     
==========================================
+ Hits        65108    65137      +29     
+ Misses       8280     8272       -8     

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[markotoplak]

I was able to reproduce the error locally by, khm, running tests from a separate directory. :) I hope now some of them will finally work.

[janezd]

I was able to reproduce the error locally by, khm, running tests from a separate directory. :) I hope now some of them will finally work.

🙈