Many Features/Tweaks/Improvements (MacOS-centric)

[BrianPugh]

hey @hrvach! This is my develop branch; it implements a bunch of feature. Cleary this is too much for a single PR; if you can identify which (if/any/all!) features you are interested in merging in, I can take the effort to isolate them to their own PR. The commits are pretty clean, so it wouldn't be too much work.

1. Improvement: Native Browser GZip Decompression to reduce config.htm size (use native browser gzip decompression #309)
2. Improvement: When flashing firmware, if the version of the 2 flashed firmwares are the same, but the CRC of the two devices differ, have device B update from A. During development, this allows for just flashing to A and it will automatically sync over to B.
3. Tweak: When entering config mode, instead of default to the device that the keyboard is plugged into, default to the currently active device. This is what I prefer, but not sure if there was an explicit reason to not do this.
4. Bugfix (MacOS): When switching between virtual desktops on macos, use the current pointer's y-coordinate instead of the center of the display. The issue with using the center-y position is that if the secondary monitor isn't arranged to overlap with the center of the primary display, it won't result in a successful screen transition.
5. Feature (Mainly MacOS, but also other OS's): Add bidirectional screen transition config/logic. Basically, we define 4 coordinates per screen transition (top/bottom of left monitor and of right monitor). When transitioning, we remap the Y-coordinate appropriately. This allows for smooth cursor movement between "jagged" arranged displays. In my case, this is between my primary monitor and my laptop monitor.
   • This blocks cursor movement between screens if outside of this border.
   • This has the additional feature of allowing "corners". I have two primary monitors that are perfectly level. Commonly, we WANT to run into corners at the top/bottom of the screen to register ourselves for selecting items from toolbars or other. By setting the borders to [[2000, 30767], [2000, 30767]] for the transition from Computer A<->B, we get this effect.
   • I was also able to simplify some of the inter-computer code with this, so there's actually not too many additional lines of code to maintain.
   • This still uses the same RightShift + F12 + Y hotkey; we determine which value to update based on screen index, cursor position, and the currently saved values.
   • TL;DR: Ultimately, this basically mimics native cursor handling in macos.
6. Feature (MacOS): Reset cursor position to Screen1 when transitioning to that computer. If the cursor was previously moved to another screen from an external mouse, it would be glitchy and out of sync.
7. Feature: Ability to Save/Upload configuration as JSON from the Web UI.
8. Bugfix (MacOS): Remapped the HID Interface numbers so that there are no collisions. Previously, in Config Mode, the relative-mouse was effectively disabled. This made MacOS screen-transitions broken whenever we are in Config Mode.
9. Improvement: Add device_t.mouse_connected field (parallels device_t.keyboard_connected).
10. Bugfix (iPad): Send all clicks via the relative-position mouse interface. For some reason, iPad does not accept clicks from an absolute-positioning device (but does accept the cursor movement). For all other devices/OS's, it seems like there's no downside to sending clicks over the relative-positioning-mouse, so I figured it's 100% a win, so no configuration option is required.
11. BugFix: It seems like the "Force Mouse Boot Mode" wasn't actually used/implemented. I think I implemented it properly; but I didn't test it heavily.
12. Feature: Add Usage Tips to the webui.
13. Feature: Make it so that the swap-outputs hotkey (LeftCtrl + CapsLock) only swap temporarily if it's press-and-held. The duration to differentiate between a tap (toggle) and a hold (temporary swap until release) is configurable through the webui. Example use-case: let's say you just want to quickly change the volume on your other computer; you can add this in to your key-presses.
14. Tweak: During firmware upgrade; keep the capslock light on. The firmware upgrade process takes like 20 seconds, and it makes it easier to identify when it's inprogress-complete. Previously it would just briefy flash when complete, but I'm not constantly staring at the LED to reliably see that.
15. Feature: When launching the config webui, automatically connect and read the deskhop device.
16. BugFix Increase the number of devices (3->4) and interfaces (6->12). Preivously I was unable to connect my QMK mouse due to there not being enough interfaces. This gives a bit more headroom at the cost of RAM (uses 23KB more ram now), but we're still only using 50% of the pi pico's ram, so it's not really a concern. The MAX_DEVICES/MAX_INTERFACES bounds check were also previously wrong. The previous code thought that dev_addr was 0-indexed, but it's effectively 1-indexed. The opposite is true for the instance.
17. BugFix (macos) Fix dragging windows between virtual desktops.
18. Add support for vertical computers/monitors

[hrvach]

Thank you very much for working on this! I am DDOS-ed with work, but I'll try finding some time and going through these changes and merging them. Really appreciate the effort and the help!

[BrianPugh]

@hrvach would it help if I were to break up this PR? Or would you consider accepting it as whole? I could start cherry-picking some of the "obvious wins" so that we can rebase this PR and gradually make it smaller and smaller.

[hrvach]

@hrvach would it help if I were to break up this PR? Or would you consider accepting it as whole? I could start cherry-picking some of the "obvious wins" so that we can rebase this PR and gradually make it smaller and smaller.

Breaking it up would be awesome for easier review, I hope this weekend I'll finally have a few hours of peace to focus on this. Apologies for being insanely slow, work is ... just nuts.

[BrianPugh]

I extracted out the commits that could just be cherry-picked into #309, #315, #316, #317, #318. #319. The other ones will require slightly more effort and are a bit more order-dependent and will require more careful interaction/planning (e.g. messing with the hole config-system and the pre-made FAT12 image). But this should get the ball rolling!

[BrianPugh]

@hrvach thanks for merging in some of the PRs! For a workflow, are we fine with me rebasing this branch off of main as we keep going? The commit history is pretty clean, so this would preserve it to make it easier to further break up this PR.

A lot of the remaining commits interact with the config stuff, so the filesystem image would need to be rebuilt. We would need to determine what we should do for those PRs to avoid merge conflicts.

[hrvach]

Sure, and once again - thank you for working on this! I built it and put as a pre-release 0.77, I'd appreciate some testing and feedback on stability.

#317 sounds reasonable, but I'm a bit uneasy to merge it without a more extensive testing on different operating systems, so maybe we could put it in e.g. development branch.

I need to test out the decompress PR as well - the initial decision to avoid depending on browser was done with hope Firefox might come around and start supporting webusb/webhid, but since that seems unlikely, I am all for simplifying it.

[hrvach]

For the tweak 3 from your list - the original reasoning is a bit silly. I wanted to make it 100% predictable and impossible to accidentally attempt to identify as usb mass storage to e.g. a work machine where such actions might be frowned upon.