Rework input_events API and expose KeyCharacterMap bindings

With the way events are delivered via an `InputQueue` with
`NativeActivity` there is no direct access to the underlying KeyEvent
and MotionEvent Java objects and no `ndk` API that supports the
equivalent of `KeyEvent.getUnicodeChar()`

What `getUnicodeChar` does under the hood though is to do lookups into a
`KeyCharacterMap` for the corresponding `InputDevice` based on the
event's `key_code` and `meta_state` - which are things we can do via
some JNI bindings for `KeyCharacterMap`.

Although it's still awkward to expose an API like
`key_event.get_unicode_char()` we can instead provide an API that
lets you look up a `KeyCharacterMap` for any `device_id` and
applications can then use that for character mapping.

This approach is also more general than the `getUnicodeChar` utility
since it exposes other useful state, such as being able to check what
kind of keyboard input events are coming from (such as a full physical
keyboard vs a virtual / 'predictive' keyboard)

For consistency this exposes the same API through the game-activity
backend, even though the game-activity backend is technically able to
support unicode lookups via `getUnicodeChar` (since it has access to the
Java `KeyEvent` object).

This highlighted a need to be able to use other `AndroidApp` APIs while
processing input, which wasn't possible with the `.input_events()` API
design because the `AndroidApp` held a lock over the backend while
iterating events.

This changes `input_events()` to `input_events_iter()` which now returns
a form of lending iterator and instead of taking a callback that gets
called repeatedly by `input_events()` a similar callback is now passed
to `iter.next(callback)`.

The API isn't as ergonomic as I would have liked, considering that
lending iterators aren't a standard feature for Rust yet but also since
we still want to have the handling for each individual event go via a
callback that can report whether an event was "handled". I think the
slightly awkward ergonomics are acceptable though considering that
the API will generally be used as an implementation detail within
middleware frameworks like Winit.

Since this is the first example where we're creating non-trivial Java
bindings for an Android SDK API this adds some JNI utilities and
establishes a pattern for how we can implement a class binding.

It's an implementation detail but with how I wrote the binding I tried
to keep in mind the possibility of creating a procmacro later that would
generate some of the JNI boilerplate involved.

Author: rib

android-activity/CHANGELOG.md

@@ -1,12 +1,108 @@
-<!-- markdownlint-disable MD022 MD024 MD032  -->
+<!-- markdownlint-disable MD022 MD024 MD032 MD033  -->
 
 # Changelog
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ## [Unreleased]
+
+### Added
+- Added `KeyEvent::meta_state()` for being able to query the state of meta keys, needed for character mapping ([#102](https://github.com/rust-mobile/android-activity/pull/102))
+- Added `KeyCharacterMap` JNI bindings to the corresponding Android SDK API ([#102](https://github.com/rust-mobile/android-activity/pull/102))
+- Added `AndroidApp::device_key_character_map()` for being able to get a `KeyCharacterMap` for a given `device_id` for unicode character mapping ([#102](https://github.com/rust-mobile/android-activity/pull/102))
+
+    <details>
+    <summary>Click here for an example of how to handle unicode character mapping:</summary>
+
+    ```rust
+    let mut combining_accent = None;
+    // Snip
+
+
+    let combined_key_char = if let Ok(map) = app.device_key_character_map(device_id) {
+        match map.get(key_event.key_code(), key_event.meta_state()) {
+            Ok(KeyMapChar::Unicode(unicode)) => {
+                let combined_unicode = if let Some(accent) = combining_accent {
+                    match map.get_dead_char(accent, unicode) {
+                        Ok(Some(key)) => {
+                            info!("KeyEvent: Combined '{unicode}' with accent '{accent}' to give '{key}'");
+                            Some(key)
+                        }
+                        Ok(None) => None,
+                        Err(err) => {
+                            log::error!("KeyEvent: Failed to combine 'dead key' accent '{accent}' with '{unicode}': {err:?}");
+                            None
+                        }
+                    }
+                } else {
+                    info!("KeyEvent: Pressed '{unicode}'");
+                    Some(unicode)
+                };
+                combining_accent = None;
+                combined_unicode.map(|unicode| KeyMapChar::Unicode(unicode))
+            }
+            Ok(KeyMapChar::CombiningAccent(accent)) => {
+                info!("KeyEvent: Pressed 'dead key' combining accent '{accent}'");
+                combining_accent = Some(accent);
+                Some(KeyMapChar::CombiningAccent(accent))
+            }
+            Ok(KeyMapChar::None) => {
+                info!("KeyEvent: Pressed non-unicode key");
+                combining_accent = None;
+                None
+            }
+            Err(err) => {
+                log::error!("KeyEvent: Failed to get key map character: {err:?}");
+                combining_accent = None;
+                None
+            }
+        }
+    } else {
+        None
+    };
+    ```
+
+    </details>
+
 ### Changed
 - GameActivity updated to 2.0.2 (requires the corresponding 2.0.2 `.aar` release from Google) ([#88](https://github.com/rust-mobile/android-activity/pull/88))
+- `AndroidApp::input_events()` is replaced by `AndroidApp::input_events_iter()` ([#102](https://github.com/rust-mobile/android-activity/pull/102))
+
+    <details>
+    <summary>Click here for an example of how to use `input_events_iter()`:</summary>
+
+    ```rust
+    match app.input_events_iter() {
+        Ok(mut iter) => {
+            loop {
+                let read_input = iter.next(|event| {
+                    let handled = match event {
+                        InputEvent::KeyEvent(key_event) => {
+                            // Snip
+                        }
+                        InputEvent::MotionEvent(motion_event) => {
+                            // Snip
+                        }
+                        event => {
+                            // Snip
+                        }
+                    };
+
+                    handled
+                });
+
+                if !read_input {
+                    break;
+                }
+            }
+        }
+        Err(err) => {
+            log::error!("Failed to get input events iterator: {err:?}");
+        }
+    }
+    ```
+
+    </details>
 
 ## [0.4.3] - 2022-07-30
 ### Fixed

android-activity/Cargo.toml

@@ -38,13 +38,15 @@ native-activity = []
 log = "0.4"
 jni-sys = "0.3"
 cesu8 = "1"
+jni = "0.21"
 ndk = "0.7"
 ndk-sys = "0.4"
 ndk-context = "0.1"
 android-properties = "0.2"
 num_enum = "0.6"
 bitflags = "2.0"
 libc = "0.2"
+thiserror = "1"
 
 [build-dependencies]
 cc = { version = "1.0", features = ["parallel"] }

android-activity/LICENSE

@@ -1,12 +1,24 @@
-The third-party glue code, under the native-activity-csrc/ and game-activity-csrc/ directories
-is covered by the Apache 2.0 license only:
+# License
 
-Apache License, Version 2.0 (docs/LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
+## GameActivity
 
+The third-party glue code, under the game-activity-csrc/ directory is covered by
+the Apache 2.0 license only:
+
+Apache License, Version 2.0 (docs/LICENSE-APACHE or <http://www.apache.org/licenses/LICENSE-2.0>)
+
+## SDK Documentation
+
+Documentation for APIs that are direct bindings of Android platform APIs are covered
+by the Apache 2.0 license only:
+
+Apache License, Version 2.0 (docs/LICENSE-APACHE or <http://www.apache.org/licenses/LICENSE-2.0>)
+
+## android-activity
 
 All other code is dual-licensed under either
 
-* MIT License (docs/LICENSE-MIT or http://opensource.org/licenses/MIT)
-* Apache License, Version 2.0 (docs/LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
+- MIT License (docs/LICENSE-MIT or <http://opensource.org/licenses/MIT>)
+- Apache License, Version 2.0 (docs/LICENSE-APACHE or <http://www.apache.org/licenses/LICENSE-2.0>)
 
-at your option.
+at your option.

android-activity/src/error.rs

@@ -0,0 +1,58 @@
+use thiserror::Error;
+
+#[derive(Error, Debug)]
+pub enum AppError {
+    #[error("Operation only supported from the android_main() thread: {0}")]
+    NonMainThread(String),
+
+    #[error("Java VM or JNI error, including Java exceptions")]
+    JavaError(String),
+
+    #[error("Input unavailable")]
+    InputUnavailable,
+}
+
+pub type Result<T> = std::result::Result<T, AppError>;
+
+// XXX: we don't want to expose jni-rs in the public API
+// so we have an internal error type that we can generally
+// use in the backends and then we can strip the error
+// in the frontend of the API.
+//
+// This way we avoid exposing a public trait implementation for
+// `From<jni::errors::Error>`
+#[derive(Error, Debug)]
+pub(crate) enum InternalAppError {
+    #[error("A JNI error")]
+    JniError(jni::errors::JniError),
+    #[error("A Java Exception was thrown via a JNI method call")]
+    JniException(String),
+    #[error("A Java VM error")]
+    JvmError(jni::errors::Error),
+    #[error("Input unavailable")]
+    InputUnavailable,
+}
+
+pub(crate) type InternalResult<T> = std::result::Result<T, InternalAppError>;
+
+impl From<jni::errors::Error> for InternalAppError {
+    fn from(value: jni::errors::Error) -> Self {
+        InternalAppError::JvmError(value)
+    }
+}
+impl From<jni::errors::JniError> for InternalAppError {
+    fn from(value: jni::errors::JniError) -> Self {
+        InternalAppError::JniError(value)
+    }
+}
+
+impl From<InternalAppError> for AppError {
+    fn from(value: InternalAppError) -> Self {
+        match value {
+            InternalAppError::JniError(err) => AppError::JavaError(err.to_string()),
+            InternalAppError::JniException(msg) => AppError::JavaError(msg),
+            InternalAppError::JvmError(err) => AppError::JavaError(err.to_string()),
+            InternalAppError::InputUnavailable => AppError::InputUnavailable,
+        }
+    }
+}

android-activity/src/game_activity/input.rs

@@ -16,7 +16,7 @@
 use num_enum::{IntoPrimitive, TryFromPrimitive};
 use std::convert::TryInto;
 
-use crate::game_activity::ffi::{GameActivityKeyEvent, GameActivityMotionEvent};
+use crate::activity_impl::ffi::{GameActivityKeyEvent, GameActivityMotionEvent};
 use crate::input::{Class, Source};
 
 // Note: try to keep this wrapper API compatible with the AInputEvent API if possible
@@ -1274,6 +1274,12 @@ impl<'a> KeyEvent<'a> {
         action.try_into().unwrap()
     }
 
+    #[inline]
+    pub fn action_button(&self) -> KeyAction {
+        let action = self.ga_event.action as u32;
+        action.try_into().unwrap()
+    }
+
     /// Returns the last time the key was pressed.  This is on the scale of
     /// `java.lang.System.nanoTime()`, which has nanosecond precision, but no defined start time.
     ///