Update use of input display

[petebankhead]

Draft PR to explore the changes introduces in qupath/qupath-fxtras#63

If accepted, then a new release of qupath-fxtras should be made (this currently uses a snapshot).

The main change is that the input display now makes key presses visible for longer, having them gentle fade out rather than disappear immediately after keys are released.

This is actually really fiddly to try to get behaving well, because a combination like Ctrl + Shift + C corresponds to multiple events, and so there needs to be some kind of 'memory' to avoid retaining only the last key that was released.

Changes to behavior would need to be made in qupath-fxtras.

There are also some new options for InputDisplay, which can be controlled via scripting:

guiscript=true

var id = getQuPath().getInputDisplay()
id.show()

// New properties
id.setShowSymbols(false) // Unicode keyboard symbols (where known)
id.setFadeTo(0.5)        // Opacity of keypresses after fading
id.setFadeDuration(javafx.util.Duration.millis(1000)) // Duration of fade
id.setSkipTextInputControls(true) // Don't show keypresses in text controls
                                  // (the script editor ignores this...)

[alanocallaghan]

I really, really like how this should work, but... when I open it and hit Ctrl+L, I just see Ctrl. Other modifiers work as expected.

I notice that Super (Windows key/apple key, I think...?) doesn't show up as a modifier. I guess it's unlikely we ever handle this, but just noting it.

[petebankhead]

I really, really like how this should work, but... when I open it and hit Ctrl+L, I just see Ctrl. Other modifiers work as expected.

Hmmm, Ctrl+L (Windows) and Cmd+L (Mac) and working for me as expected - but I haven't tried Linux.

Do you mean Ctrl+L specifically fails, or Ctrl fails no matter what letter it is paired with? If it's just Ctrl+L then I guess it's a tolerable (and perhaps Linux-specific?) window-focussing issue.

Windows key probably just needs to be added here.

[alanocallaghan]

I think we shouldn't use the system/super key anyway, as traditionally it is for system-level shortcuts, and we don't currently support user keymaps.

The control issue is odd, and yes it is for every key along with Ctrl. The line

currentKeys.put(code, getText(event));

collects normal text when alt/shift/etc are depressed, but when ctrl is down, the text gets mangled, probably by some Linux quirk, so the current key map looks like this:

{A=�}

or for multiple keys:

{A=�, D=�}

This then gets displayed as (what looks like?) a blank space in the input display, or for multiple keys as blank spaces with + between.

I'll try solve this as I suspect you are not as keen to understand Linux weirdness as I am

[petebankhead]

Thanks, I’m not sure what JavaFX KeyCode the system/super key would correspond to. There’s some fun, platform-specific shenanigans in the JavaFX code to do with converting modifiers, but it doesn’t seem straightforward for us to use it directly… so there’s some awkward custom logic in qupath-fxtras presumably could/should be better.

[alanocallaghan]

I think Super == Windows so should be alright

[petebankhead]

I've just updated the snapshot of qupath-fxtras to include the fixes from qupath/qupath-fxtras#64 - so if you rebuild this (possibly with --refresh-dependencies) then hopefully it's better.

[alanocallaghan]

Oh thanks, I had a local snapshot of fxtras on my local maven and ran that with qupath; is there an easier way? I think I tried includeBuild before with not a lot of luck

[petebankhead]

I was using includeBuild successfully here. You’ll need that or your strategy to check any additional changes, but since our PR is now in the latest maven snapshot then using ./gradlew run --refresh-dependencies should be enough to test it (and --refresh-dependencies might not be needed).

[alanocallaghan]

Looks good, but num lock for me has this value (should be 144), which is "not blank" but does show up as an invisible block:

Could modify the if statement that returns code.getName(); to guard against invalid chars generally, but I'm tempted to ignore this for the time being as the numbers still show up on the input display with num lock on...

        private String getText(KeyCode code) {
            if (code.isLetterKey())
                return code.getName().toUpperCase();
            else {
                if (showSymbolsProperty.get()) {
                    var symbol = getSymbol(code);
                    if (symbol != null)
                        return symbol;
                }
                if (code.isModifierKey() || code.getChar().isBlank() || code.isFunctionKey())
                    return code.getName();
                else
                    return code.getChar();
            }
        }

[petebankhead]

I should probably have stayed away from num lock since Macs don't have a dedicated key for it, and I don't have a suitable keyboard to test it with in Windows.

Made some more changes at qupath/qupath-fxtras#65

Could modify the if statement that returns code.getName(); to guard against invalid chars generally

Sounds desirable, although I've no idea how to do it reliably. Feel free to make a PR or approve if you think it's currently good enough to use for the release.

@finglis also please check it out.

[alanocallaghan]

I'm not really a fan of the Home/End/Page Up/Page Down symbols, and if I run:

var id = getQuPath().getInputDisplay()
id.setShowSymbols(false)

then for Home/End/Page Up/Page Down I get $/#/!/" respectively, and for up/down/left/right I get &/(/%/'.

[petebankhead]

Can you make a PR to change them? I don't mind using fewer symbols and more text if that's safer - and/or switching to default to not using symbols.

[Rylern]

Looks good on Linux (Fedora) and MacOS

[finglis]

Some issues pointed out in call:

1. Windows specific - scroll arrow remains even after release (but will eventually randomly disappear)
2. When pressing a combo (eg ctrl + shift + c) and then pressing another keyboard key after. It looks like the ctrl + shift was pressed in combination with it instead (eg v on its own looks like ctrl + shift + v).
3. Shift + numbers will not show the punctuation key that corresponds, only that shift + number has been pressed. Possibly not ideal for cross platform demonstrating since shift +2 is not the same output on windows and mac.