User gestures

[AshleyScirra]

Description

User gestures (aka user activation) restrict certain sensitive actions, such as opening a popup window, to only be allowed in response to a user gesture, i.e. some kind of user input event. Browsers currently significantly differ in their implementation of this though, and it causes awkward compatibility problems.

Rationale

The model implemented by Chrome works well: there are essentially two flags and a short timeout. Other browsers do not support this though, only considering synchronous code run inside a user input event as a user gesture.

This causes problems such as:

• difficulty running async code for input events (an await can mean you lose the gesture)
• difficulty delegating work to Web Workers (postMessage can mean you lose the gesture)
• blocking use of new APIs like OffscreenCanvas (in a web worker, you never have a user gesture)
• causing other browsers to implement case-by-case exceptions e.g. allowing user gestures to propagate with certain APIs, but not others; or modifying the design of APIs (e.g. Clipboard API) to solve a similar problem

Chrome's model solves all these problems, assuming there is not too long a wait. However in many cases that is perfectly sufficient. For example converting a Blob to an ArrayBuffer is async but will likely complete very quickly for small blobs; in Chrome a user gesture can still be used afterwards, but in other browsers it cannot.

Specification

"Tracking user activation" in the HTML spec: https://html.spec.whatwg.org/multipage/interaction.html#tracking-user-activation

Tests

https://wpt.fyi/results/html/user-activation

[foolip]

Thanks for proposing this, @AshleyScirra!

blocking use of new APIs like OffscreenCanvas (in a web worker, you never have a user gesture)

Can you say more about this? Are there any OffscreenCanvas APIs that require a user gesture?

cc @mustaqahmed who's done a lot of work on user gestures in Chrome.

[AshleyScirra]

We have a game engine that supports running entirely in a web worker with OffscreenCanvas. In our architecture input events are forwarded to the worker via postMessage(), it runs the entire engine and all logic in the worker, and then it posts back for calls to any APIs not available in a worker (window.open, WebRTC, etc...)

However this means calls are never synchronously in a user input event, and so in some browsers we lose the ability to use any user-gesture limited APIs at all. So we have to turn off using a web worker and stop using OffscreenCanvas, just so we can keep within the user gesture rules. I know OffscreenCanvas isn't supported outside of Chromium yet, but even when it is, this is a blocking issue for us to switch to using it.

[foolip]

I see, so an example might be when there's a "fullscreen" button in the game's UI, and you need to actually run the game engine to know that it was clicked, and then do the element.requestFullscreen() call on the main thread. Is that the kind of thing that's made complicated by the model in some browsers?

[mustaqahmed]

To the best of my knowledge, most other browsers use the old HTML model that required calling activation-gated APIs synchronously in appropriate event handlers, or asynchronously after a (undefined) propagation. That undefined part was known to have fundamental problems, and every browser (including pre-M72 Chrome) added unpredictable workarounds to make it work. The primary motivation for the current HTML model was to support async usage of user activation in a well-defined manner that is agnostic to any propagation scheme in JS.

[AshleyScirra]

I see, so an example might be when there's a "fullscreen" button in the game's UI, and you need to actually run the game engine to know that it was clicked, and then do the element.requestFullscreen() call on the main thread. Is that the kind of thing that's made complicated by the model in some browsers?

Yes, exactly. It might seem a roundabout approach to making the call, but if you make middleware/game engines/libraries that run in a worker, you can end up with that situation; by the time you try to call requestFullscreen() you've lost the user gesture. I suppose a similar problem may arise with trying to host WebAssembly builds of codebases in a worker. The only good workaround is to stop using a worker and run everything on the main thread again.

I guess this is a variant of "difficulty delegating work to Web Workers", but I called it out separately as giving up on running code off main thread is a pretty significant consequence.

[tbondwilkinson]

User gestures is a P2 for Google's closure library

"Allows more work to be done asynchronously, which unblocks migration to Promise."

[marcoscaceres]

WebKit implements the "new" model in HTML, and has for a long time.

About the problem @AshleyScirra outlines, understood. I guess it would be good to see some examples. WebKit, for instance, allows quite a bit of time before consuming the activation (this obviously shouldn't be relied on and can change without notice!).

So, I guess I'm wondering why one would do:

1. User clicks.
2. Send message to worker.
3. await response
4. requestFullscreen()

Instead of:

1. User clicks.
2. requestFullscreen() + do anything else that requires transient activation.
3. Send message to worker.
4. await response
5. exit full screen if something went wrong.