Move `can_create_surfaces()`/`destroy_surfaces()` from `ApplicationHandler` to `WindowEvent`

[MarijnS95]

It has been brought up in many issues over the years, that winit to this day doesn't support multiple Activitys on Android. I've always envisioned them to be analogous to a system-created window (i.e. needing a new ApplicationHandler::window_created(new_window: Window) callback of sorts) that the user could possibly launch themselves asynchronously with https://github.com/rust-mobile/android-intent.

As brought up again in #4419 this requires the surface events to be moved to WindowEvent as they're strictly related to a single Activity (thus winit Window) for which the underlying presentation surface (a layer in the composition stack/tree of sorts) is created or destroyed.

Looking through the current examples, it completely eludes me why can_create_surfaces() has been the selected place to create new windows now, as that is entirely counter to what the original Event::Resumed used/intended to represent. On winit's Android backend, a Window is anyway a bogus concept that doesn't really represent anything right now.

Is that something we can move and resolve before winit 0.31 is finalized and released, allowing Android to possibly "catch up" without breaking core/API releases - and if anything to demonstrate the right semantics in at least our examples/documentation/guidelines?

Related downstream confusion

• examples: Update winit requirement to 0.30 ash-rs/ash#1008 (comment)

[kchibisov]

It has been brought up in many issues over the years, that winit to this day doesn't support multiple Activitys on Android. I've always envisioned them to be analogous to a system-created window (i.e. needing a new ApplicationHandler::window_created(new_window: Window) callback of sorts) that the user could possibly launch themselves asynchronously with https://github.com/rust-mobile/android-intent.

This is the end design I want to kind of have as well. Like we try to solve an issue that Window is not ready, in this case we should indeed, move the can_create_surfaces to be a WindowEvent, since window is ready before, but at this point, why can't we just delay the entire window itself, since it's not useful (As I was suggesting)? We have similar issue with Wayland, but it's solved there internally.

[madsmtm]

I fully agree that can_create_surfaces()/destroy_surfaces() should be window events.

I actually tend to think that they should be WindowEvent::Created / WindowEvent::Destroyed, since that is what happens on Android IIUC? The entire window is torn down and re-created (with state restoration) whenever you do something as simple as rotate your phone (though you can opt out of it in some cases by modifying the manifest).

[kchibisov]

I actually tend to think that they should be WindowEvent::Created / WindowEvent::Destroyed

I think we can start with something like that actually, and then move to proper event that sends you the window.

[MarijnS95]

I actually tend to think that they should be WindowEvent::Created / WindowEvent::Destroyed, since that is what happens on Android IIUC? The entire window is torn down and re-created (with state restoration) whenever you do something as simple as rotate your phone (though you can opt out of it in some cases by modifying the manifest).

"Unfortunately" not, you're conflating two things here.

When an app doesn't opt into any so-called configuration changes, yes your Activity (which I want to become analogous to a Window) is entirely torn down and recreated, see onCreate and onDestroy in the lifecycle: https://developer.android.com/guide/components/activities/activity-lifecycle.

But that's very inefficient, which is why Android allows configuring to which configChanges the app can respond which are delivered via simple callbacks.

---

And both of those are "entirely" decoupled from Surface lifetimes. While these definitely occur when the Activity is (re)started or resized, they also happen when the user navigates away from the Activity or locks their phone where the compositor will "immediately" remove the render surface even if the Activity or Window as it were are still alive (only paused/stopped or resumed/started).

---

TL;DR: On Android Activity (Window) lifetime != Surface lifetime. We can't tie can_create/destroy_surfaces() to Window::Created/Destroyed events, they're separate things.

[nicoburns]

I've been upgrading Blitz to Winit 0.31, and wanted to add my support for this change. This event doesn't make much sense to me where it is but seems like a natural fit for a WindowEvent (and would actually clarify when I should be creating/destoying surfaces which was never entirely clear to me in the 0.30 API).

Would we be able to maintain the invariant that CanCreateSurfaces/DestroySurfaces event pairs always occur between Created/Destroyed event pairs (across all platforms)?

[MarijnS95]

@nicoburns yes, I think that invariant should hold (if/how winit should validate it - possibly in debug mode - is not known to me). Otherwise receiving either of those events for an inexistent WindowId (before Created or after Destroyed), like any other WindowEvent, seems completely borked.

EDIT: It seems @madsmtm was writing such validation in #3710

Related: we already have #3206 / #4419 to ensure DestroySurfaces will also be raised on all platforms, as right now that is only true for CanCreateSurfaces.

[nicoburns]

@MarijnS95 Follow-up question. I've just realised that can_create_surfaces is now called instead of resume (which is no longer called at application startup (at least on macOS)). I am currently deferring creation of windows until can_create_surfaces (used to be resume) is called. That obviously wouldn't work if can_create_surfaces was a WindowEvent as a window to receieve the event would never be created!

Does this mean that it is now ok to create windows immediately in my main() function. Or is there is there some other event I should be waiting for to create windows?

[MarijnS95]

@nicoburns good point, see my original issue description: I was also very confused by seeing window creation inside can_create_surfaces(), which - for the purposes of Android - is a feature "on" a window.

It's unclear to me where these now belong though. In Android, those suspend/resume events are also part of an Activity, which I wanted to make analogous to a Window, so that won't work.

Personally, perhaps it should be async? Have an (Active)EventLoop and call create_window() for which later WindowEvent::Created is raised?

[nicoburns]

In Android, those suspend/resume events are also part of an Activity, which I wanted to make analogous to a Window, so that won't work.

@MarijnS95

I've been looking into this a bit more, and I feel like a Window on Android ought to be a View not an Activity. Which would make it similar to iOS where a Window is a UIView. Also similarly to iOS where it is now (in Winit 0.31) possible for the end-user to own the UIApplicationDelegate, this could also allow users to own the Activity and implement it using their own subclass.

I believe this would this approach would allow for a lot of interesting functionality such as creating an app "shell" using the native android UI toolit into which 1 or many Winit Windows are embedded (for apps with a more native feel), or embedding a winit app as a single view in an existing native android app.

It looks to me that https://github.com/rust-mobile/android-view has most of the code to implement this. Specifically (because it took me a while to workout this repo) https://github.com/rust-mobile/android-view/tree/main/library/src/main/java/org/linebender/android/rustview is the Java side (Android SurfaceView and InputConnection subclasses as a library) and https://github.com/rust-mobile/android-view/tree/main/src is the Rust side (FFI to recieve events from the Java classes + a Rust trait for rust code to hook into those events).

The code to use this (that would end up in a lot end-user projects) is then very minimal: https://github.com/rust-mobile/android-view/blob/main/app/src/main/java/org/linebender/android/viewdemo/DemoActivity.java

Thoughts?

[MarijnS95]

I've written my own code against ASurface(Control) and the new AInputReceiver, which fundamentally turns out to be a completely different backend from Rust being "in control" over some sort of Activity.

In the context of NativeActivity and other custom Activity subclasses the lifetime of the "root Surface" doesn't at all line up with what I expect from a Window, making the proposed Subsurface abstraction for Winit a better fit.

EDIT 3 weeks later: hidden both comments as they provide zero context or evidence against moving surface lifetime callbacks to WindowEvents. This applies just as well when a winit::Window wraps an Android View (which is what a NativeActivity channels to some extent as well).