Skip to content

Commit 9279037

Browse files
mingycmingyc
authored
[explainer] Reorganize explainer (#79)
- Added back explainer for PendingBeacon API from #69 for future reference. - Simplified the contents in `README.md`. API details are moved to fetch-with-pending-request-api.md. - Added empty fetch-later-api.md. Subsequent changes will fill in details according to the PR in fetch repository. - Added a Github Action to verify links in markdown files.
1 parent d5189eb commit 9279037

File tree

8 files changed

+731
-384
lines changed

8 files changed

+731
-384
lines changed

‎.github/workflows/linter.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,9 @@ name: Super Linter
77

88
on:
99
push:
10-
branches: [ "main" ]
10+
branches: [main]
1111
pull_request:
12-
branches: [ "main" ]
12+
branches: [main]
1313

1414
jobs:
1515
run-lint:

‎.github/workflows/md-lint.yml

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
name: Check Markdown links
2+
3+
#
4+
# Documentation:
5+
# https://github.com/gaurav-nelson/github-action-markdown-link-check#how-to-use
6+
#
7+
8+
on:
9+
push:
10+
branches: [main]
11+
paths: ["**.md"]
12+
pull_request:
13+
branches: [main]
14+
paths: ["**.md"]
15+
16+
jobs:
17+
markdown-link-check:
18+
runs-on: ubuntu-latest
19+
steps:
20+
- uses: actions/checkout@v3
21+
- uses: gaurav-nelson/github-action-markdown-link-check@v1

‎README.md

Lines changed: 42 additions & 382 deletions
Large diffs are not rendered by default.

‎docs/alternative-approaches.md

Lines changed: 169 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
# Alternative Approaches
2+
3+
*This document maintains the list of alternatives that have been considered to achieve better beaconing.*
4+
*Note that only [PendingBeacon API](#pendingbeacon-api) has ever been implemented.*
5+
6+
## fetch() with PendingRequest API
7+
8+
See the [fetch() with PendingRequest API explainer](fetch-with-pending-request-api.md).
9+
10+
## PendingBeacon API
11+
12+
See the [PendingBeacon API explainer](pending-beacon-api.md).
13+
14+
## DOM-Based API
15+
16+
A DOM-based API was considered as an alternative.
17+
This API would consist of a new possible `beacon` value for the `rel` attribute
18+
on the link tag, which developers could use to indicate a beacon,
19+
and then use standard DOM manipulation calls to change the data, cancel the beacon, etc.
20+
21+
The stateful JS API was preferred to avoid beacon concerns intruding into the DOM,
22+
and because a ‘DOM-based’ API would still require scripting in many cases anyway
23+
(populating beacon data as the user interacts with the page, for example).
24+
25+
## BFCache-supported `unload`-like event
26+
27+
Another alternative is to introduce (yet) another page lifecycle event,
28+
that would be essentially the `unload` event, but supported by the BFCache -
29+
that is, its presence would not disable the BFCache, and the browser would execute this callback even on eviction from the BFCache.
30+
This was rejected because it would require allowing pages frozen in the BFCache to execute a JavaScript callback,
31+
and it would not be possible to restrict what that callback does
32+
(so, a callback could do things other than sending a beacon, which is not safe).
33+
It also doesn’t allow for other niceties such as resilience against crashes or batching of beacons,
34+
and complicates the already sufficiently complicated page lifecycle.
35+
36+
## Extending `fetch()` API
37+
38+
> **NOTE:** Discussions in [#52] and [#50].
39+
40+
Another alternative is to extend the [Fetch API] to support the [requirements](../README.md#requirements).
41+
42+
The existing Fetch with `keepalive` option, combined with `visibilitychagne` listener, can approximate part of (1):
43+
44+
```js
45+
document.addEventListener('visibilitychange', () => {
46+
if (document.visibilityState === 'hidden') {
47+
fetch('/send_beacon', {keepalive: true});
48+
// response may be dropped.
49+
}
50+
});
51+
```
52+
53+
or a new option `deferSend` may be introduced to cover the entire (1):
54+
55+
```js
56+
// defer request sending on `hidden` or bfcahce eviction etc.
57+
fetch('/send_beacon', {deferSend: true});
58+
// Promise may not resolve and response may be dropped.
59+
```
60+
61+
### Problem
62+
63+
However, there are several problem with this approach:
64+
65+
1. **The Fetch API shape is not designed for this (1) purpose.** Fundamentally, `window.fetch` returns a Promise with Response to resolve, which don't make sense for beaconing at page discard that doesn't expect to process response.
66+
2. **The (1) mechanism is too unrelated to be added to the Fetch API**. Even just with a new option, bundling it with a visibility event-specific behavior just seems wrong in terms of the API's scope.
67+
3. **The Fetch API does not support updating request URL or data.** This is simply not possible with its API shape. Users have to re-fetch if any update happens.
68+
69+
The above problems suggest that a new API is neccessary for our purpose.
70+
71+
## Extending `navigator.sendBeacon()` API
72+
73+
> **NOTE:** Discussions in [WebKit's standard position](https://github.com/WebKit/standards-positions/issues/85#issuecomment-1418381239).
74+
75+
Another alternative is to extend the [`navigator.sendBeacon`] API:
76+
77+
```ts
78+
navigator.sendBeacon(url): bool
79+
navigator.sendBeacon(url, data): bool
80+
```
81+
82+
To meet the [requirements](../README.md#requirements) and to make the new API backward compatible, we propose the following shape:
83+
84+
```ts
85+
navigator.sendBeacon(url, data, fetchOptions): PendingBeacon
86+
```
87+
88+
An optional dictionary argument `fetchOptions` can be passed in, which changes the return value from `bool` to `PendingBeacon` proposed in the [above section](#pendingbeacon-api). Some details to note:
89+
90+
1. The proposal would like to support both `POST` and `GET` requests. As the existing API only support `POST` beacons, passing in `fetchOptions` with `method: GET` should enable queuing `GET` beacons.
91+
2. `fetchOptions` can only be a subset of the [Fetch API]'s [`RequestInit`] object:
92+
1. `method`: one of `GET` or `POST`.
93+
2. `headers`: supports custom headers, which unblocks [#50].
94+
3. `body`: **not supported**. POST body should be in `data` argument.
95+
4. `credentials`: enforcing `same-origin` to be consistent.
96+
5. `cache`: not supported.
97+
6. `redirect`: enforcing `follow`.
98+
7. `referrer`: enforcing same-origin URL.
99+
8. `referrerPolicy`: enforcing `same-origin`.
100+
9. `keepalive`: enforcing `true`.
101+
10. `integrity`: not supported.
102+
11. `signal`: **not supported**.
103+
* The reason why `signal` and `AbortController` are not desired is that we needs more than just aborting the requests. It is essential to check a beacon's pending states and to update or accumulate data. Supporting these requirements via the returned `PendingBeacon` object allows more flexibility.
104+
12. `priority`: enforcing `auto`.
105+
3. `data`: For `GET` beacon, it must be `null` or `undefined`.
106+
4. The return value must supports updating request URL or data, hence `PendingBeacon` object.
107+
108+
### Problem
109+
110+
* The above API itself is enough for the [requirements](../README.md#requirements) (2) and (3), but cannot achieve the requirement (1), delaying the request.
111+
* The function name `sendBeacon` semantic doesn't make sense for the "delaying" behavior.
112+
* Combing the subset of `fetchOptions` along with the existing `data` parameter are error-proning.
113+
114+
## Introducing `navigator.queueBeacon()` API
115+
116+
To imprvoe from "Extending `navigator.sendBeacon()` API, it's better with a new function:
117+
118+
```ts
119+
navigator.queueBeacon(url, fetchOptions, beaconOptions): PendingBeacon
120+
```
121+
122+
This proposal gets rid of the `data` parameter, and request body should be put into `fetchOptions.body` directly.
123+
124+
The extra `beaconOptions` is a dictionary taking `backgroundTimeout` and `timeout` to support the optional timeout after bfcache or hidden requirement.
125+
126+
At the end, this proposal also requires an entirely new API, just under the existing `navigator` namespace. The advantage is that we might be able to merge this proposal into [w3c/beacon] and eliminate the burden to maintain a new spec.
127+
128+
129+
## Write-only API
130+
131+
This is similar to the [proposed PendingBeacon API](#pendingbeacon-api) but there is no `pending` and no `setData()`.
132+
There are 2 classes of beacon with a base class that has
133+
134+
* `url`
135+
* `method`
136+
* `sendNow()`
137+
* `deactivate()`
138+
* API for specifying timeouts
139+
140+
With these APIs, the page cannot check whether the beacon has been sent already.
141+
142+
It's unclear that these APIs can satisfy all use cases.
143+
If they can, they have the advantage of being easier to implement
144+
and simple to use.
145+
146+
## High-Level APIs
147+
148+
### AppendableBeacon
149+
150+
Has `appendData(data)` which appends new data to the beacon's payload.
151+
The beacon will flush queued payload according to the timeouts and the browser state.
152+
153+
The use-case is for continuously logging events that are accumulated on the server-side.
154+
155+
### ReplaceableBeacon
156+
157+
Has `replaceData(data)` which replaces the current the beacon's payload.
158+
The beacon will send the payload according to the timeouts and the browser state.
159+
If a payload has been sent already, replaceData simply stores a new payload to be sent in the future.
160+
161+
The use case is for logging a total-so-far.
162+
The server would typically only pay attention to the latest value.
163+
164+
165+
[#50]: https://github.com/WICG/pending-beacon/issues/50
166+
[#52]: https://github.com/WICG/pending-beacon/issues/52
167+
[Fetch API]: https://fetch.spec.whatwg.org/#fetch-api
168+
[`RequestInit`]: https://fetch.spec.whatwg.org/#requestinit
169+
[w3c/beacon]: https://github.com/w3c/beacon

‎docs/fetch-later-api.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
# fetchLater() API
2+
3+
4+
TODO

‎docs/fetch-with-pending-request-api.md

Lines changed: 188 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,188 @@
1+
# fetch() with PendingRequest API (deprecated)
2+
3+
**WARNING: This API is being replaced with [`fetchLater()`](fetch-later-api.md), a Fetch-based approach.**
4+
5+
--
6+
7+
*This document is an explainer for PendingRequest API.*
8+
*It is proposed in response to a series of [discussions and concerns][concerns] around the experimental [PendingBeacon API](pending-beacon-api.md).*
9+
*There many [open discussions](#open-discussions) within the explainers, which leads to the fetchLater() API.*
10+
11+
*Note that this proposal is NEVER implemented.*
12+
13+
## Design
14+
15+
> **NOTE:** Discussions in [#70], [#52] and [#50].
16+
17+
The basic idea is to extend the [Fetch API] by adding a new stateful option:
18+
Rather than a developer manually calling `fetch(url, {keepalive: true})` within a `visibilitychange` event listener, the developer registers that they would like to send a pending request, i.e. a beacon, for this page when it gets discarded.
19+
The developer can then call signal controller registered on this request to updates based on its state or abort.
20+
21+
Then, at some point later after the user leaves the page, the browser will send the request.
22+
From the point of view of the developer the exact send time is unknown. On successful sending, the whole response will be ignored, including body and headers. Nothing at all should be processed or updated, as the page is already gone.
23+
24+
## JavaScript API
25+
26+
The following new fetch options are introduced into [`RequestInit`]:
27+
28+
* `deferSend`: A `DeferSend` object. If set, the browser should defer the request sending until page discard or bfcache eviction.
29+
Underlying implementation should ensure the request is kept alive until suceeds or fails.
30+
Hence it cannot work with `keepalive: false`. The object may optionally set the following field:
31+
* `sendAfterBeingBackgroundedTimeout`: Specifies a timeout in seconds for a timer that only starts after the page enters the next `hidden` visibility state.
32+
Default to `-1`.
33+
* `sentSignal`: A `SentSignal` object to allow user to listen to the `sent` event of the request when it gets sent.
34+
35+
36+
## Examples
37+
38+
### Defer a `GET` request until page discard
39+
40+
```js
41+
fetch('/send_beacon', {deferSend: new DeferSend()}).then(res => {
42+
// Promise may never be resolved and response may be dropped.
43+
})
44+
```
45+
46+
### Defer a request until next `hidden` + 1 minute
47+
48+
```js
49+
fetch('/send_beacon', {
50+
deferSend: new DeferSend(sendAfterBeingBackgroundedTimeout: 60)
51+
}).then(res => {
52+
// Possibly resolved after next `hidden` + 1 minute.
53+
// But this may still not be resolved if page is already in bfcache.
54+
})
55+
```
56+
57+
### Update a pending request
58+
59+
```js
60+
let abort = null;
61+
let pending = true;
62+
63+
function createBeacon(data) {
64+
pending = true;
65+
abort = new AbortController();
66+
let sentSignal = new SentSignal();
67+
fetch(data, {
68+
deferSend: new DeferSend(),
69+
signal: abortController.signal,
70+
sentSignal: sentsentSignal
71+
});
72+
73+
sentSignal.addEventListener("sent", () => {
74+
pending = false;
75+
});
76+
}
77+
78+
function updateBeacon(data) {
79+
if (pending) {
80+
abort.abort();
81+
}
82+
createBeacon(data);
83+
}
84+
```
85+
86+
## Open Discussions
87+
88+
### 1. Limiting the scope of pending requests
89+
90+
> **NOTE:** Discussions in [#72].
91+
92+
Even if moving toward a fetch-based design, this proposal does still not focus on supporting every type of requests as beacons.
93+
94+
For example, it's non-goal to support most of [HTTP methods], i.e. being able to defer an `OPTION` or `TRACE`.
95+
We should look into [`RequestInit`] and decide whether `deferSend` should throw errors on some of their values:
96+
97+
* `keepalive`: must be `true`. `{deferSend: new DeferSend(), keepalive: false}` conflicts with each other.
98+
* `url`: supported.
99+
* `method`: one of `GET` or `POST`.
100+
* `headers`: supported.
101+
* `body`: only supported for `POST`.
102+
* `signal`: supported.
103+
* `credentials`: enforcing `same-origin` to be consistent.
104+
* `cache`: not supported?
105+
* `redirect`: enforcing `follow`?
106+
* `referrer`: enforcing same-origin URL?
107+
* `referrerPolicy`: enforcing `same-origin`?
108+
* `integrity`: not supported?
109+
* `priority`: enforcing `auto`?
110+
111+
As shown above, at least `keepalive: true` and `method` need to be enforced.
112+
If going with this route, can we also consider the [PendingRequest API] approach that proposes a subclass of `Request` to enforce the above?
113+
114+
### 2. `sendAfterBeingBackgroundedTimeout` and `deferSend`
115+
116+
> **NOTE:** Discussions in [#73], [#13].
117+
118+
```js
119+
class DeferSend {
120+
constructor(sendAfterBeingBackgroundedTimeout)
121+
}
122+
```
123+
124+
Current proposal is to make `deferSend` a class, and `sendAfterBeingBackgroundedTimeout` its optional field.
125+
126+
1. Should this be a standalone option in [`RequestInit`]? But it is not relevant to other existing fetch options.
127+
2. Should it be after `hidden` or `pagehide` (bfcached)? (Previous discussion in #13).
128+
3. Need user input for how desirable for this option.
129+
4. Need better naming suggestion.
130+
131+
### 3. Promise
132+
133+
> **NOTE:** Discussions in [#74].
134+
135+
To maintain the same semantic, browser should resolve Promise when the pending request is sent. But in reality, the Promise may or may not be resolved, or resolved when the page is in bfcache and JS context is frozen. User should not rely on it.
136+
137+
### 4. `SendSignal`
138+
139+
> **NOTE:** Discussions in [#75].
140+
141+
This is to observe a event to tell if a `deferSend` request is still pending.
142+
143+
To prevent from data races, the underlying implementation should ensure that renderer is authoritative to the request's send state when it's alive. Similar to [this discussion](https://github.com/WICG/pending-beacon/issues/10#issuecomment-1189804245) for PendingBeacon.
144+
145+
### 5. Handling Request Size Limit
146+
147+
> **NOTE:** Discussions in [#76].
148+
149+
As setting `deferSend` implies `keepalive` is also true, such request has to share the same size limit budget as a regular keepalive request’s [one][fetch-keepalive-quota]: "for each fetch group, the sum of contentLength and inflightKeepaliveBytes <= 64 KB".
150+
151+
To comply with the limit, there are several options:
152+
153+
1. `fetch()` throws `TypeError` whenever the budget has exceeded. Users will not be able to create new pending requests.
154+
2. The browser forces sending out other existing pending requests, in FIFO order, when the budget has exceeded. For a single request > 64KB, `fetch()` should still throws `TypeError`.
155+
3. Ignore the size limit if [BackgroundFetch] Permission is enabled for the page.
156+
157+
158+
### 6. Permissions Policy
159+
160+
> **NOTE:** Discussions in [#77].
161+
162+
Given that most reporting API providers are crossed origins, we propose to allow this feature by default for 3rd-party iframes.
163+
User should be able to opt out the feature with the corresponding Permissions Policy.
164+
165+
166+
## Other Documentation
167+
168+
* [Initial PendingRequest API Proposal](https://docs.google.com/document/d/1QQFFa6fZR4LUiyNe9BJQNK7dAU36zlKmwf5gcBh_n2c/edit#heading=h.powlqxc01y5b)
169+
* [Presntation at WebPerf WG 2023/03/16](https://docs.google.com/presentation/d/1w_v2kn4RxDmGQ76HAHbuWpYMPj7XsHkYOILIkLs9ppY/edit#slide=id.pZ)
170+
171+
172+
[concerns]: https://github.com/WICG/pending-beacon/issues/70
173+
[#13]: https://github.com/WICG/pending-beacon/issues/13
174+
[#50]: https://github.com/WICG/pending-beacon/issues/50
175+
[#52]: https://github.com/WICG/pending-beacon/issues/52
176+
[#70]: https://github.com/WICG/pending-beacon/issues/70
177+
[#72]: https://github.com/WICG/pending-beacon/issues/72
178+
[#73]: https://github.com/WICG/pending-beacon/issues/73
179+
[#74]: https://github.com/WICG/pending-beacon/issues/74
180+
[#75]: https://github.com/WICG/pending-beacon/issues/75
181+
[#76]: https://github.com/WICG/pending-beacon/issues/76
182+
[#77]: https://github.com/WICG/pending-beacon/issues/77
183+
[Fetch API]: https://fetch.spec.whatwg.org/#fetch-api
184+
[`RequestInit`]: https://fetch.spec.whatwg.org/#requestinit
185+
[fetch-keepalive-quota]: https://fetch.spec.whatwg.org/#http-network-or-cache-fetch
186+
[BackgroundFetch]: https://developer.mozilla.org/en-US/docs/Web/API/Background_Fetch_API#browser_compatibility
187+
[HTTP methods]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods
188+
[PendingRequest API]: https://docs.google.com/document/d/1QQFFa6fZR4LUiyNe9BJQNK7dAU36zlKmwf5gcBh_n2c/edit#heading=h.xs53e9immw2r

0 commit comments

Comments
 (0)