Skip to content

Commit 33a1c2d

Browse files
jcscottiiijcscottiii
authored
RFC #163: Add web_feature metadata file to web-platform-tests (#163)
* Create feature-set-metadata-file-to-web-platform-tests.md * Update feature-set-metadata-file-to-web-platform-tests.md * start to address feedback * clarify spec links * simplify * rename * finish renaming * add lint suggestion * add new jsonschema and examples * rename web_feature to web_features
1 parent 51538dc commit 33a1c2d

File tree

1 file changed

+244
-0
lines changed

1 file changed

+244
-0
lines changed

‎rfcs/feature-set-metadata-file-to-web-platform-tests.md

Lines changed: 244 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,244 @@
1+
# RFC #163: Add web_features metadata file to web-platform-tests
2+
3+
Author: @jcscottiii
4+
5+
# Introduction
6+
7+
The web-platform-tests (WPT) project is a valuable resource for testing web
8+
platform features and contains valuable metadata such as feature spec links.
9+
Recently, the [WebDX Community Group](https://www.w3.org/community/webdx/)
10+
began creating the [web-features](https://github.com/web-platform-dx/web-features)
11+
repository. That repository serves as a basic shared catalog of feature
12+
definitions of the web platform. The repository itself doesn't intend to produce
13+
data but rather link to existing data that will inform audiences which features
14+
are part of [Baseline](https://web.dev/baseline/). However, there's a great
15+
opportunity to connect the WPT ecosystem to the web-features catalog. By doing
16+
this, it would enable users of wpt.fyi to filter by web-features grouping, which
17+
is similar to
18+
[the ability to filter by spec links](https://github.com/web-platform-tests/wpt.fyi/issues/1489).
19+
The RFC proposes the addition of a WEB_FEATURES.yml metadata file which would
20+
enable the linkage between WPT and web-features.
21+
22+
# Proposed change
23+
24+
The proposed change includes:
25+
26+
1. Introduce a new metadata file, WEB_FEATURES.yml
27+
2. Add linting functionality
28+
3. Create a script to generate a manifest
29+
4. Adjust the wpt-pr-bot to handle reviews of the changes
30+
31+
Below are the details for each step.
32+
33+
## Step 1. Introduce a new metadata file, WEB_FEATURES.yml
34+
35+
Originally, there was a RFC that proposed adding these details to the existing
36+
META.yml. During the August 01, 2023 Monthly WPT
37+
[Meeting](https://github.com/web-platform-tests/wpt-notes/blob/master/minutes/2023-08-01.md#rfc-157---add-feature-meta-tag-to-web-platform-tests-meta-data),
38+
it was discussed that it should be a separate file. This section describes the structure of the file.
39+
40+
This file is expected to be in the same places developers would expect a META.yml.
41+
42+
### File examples
43+
44+
#### Example 1: Default
45+
46+
```yaml
47+
features:
48+
- name: feature1
49+
files: "**"
50+
```
51+
52+
#### Example 2: A directory which has multiple features:
53+
54+
```yaml
55+
features:
56+
- name: test-2
57+
files:
58+
- test-2-01.html
59+
- test-2-02.html
60+
- name: test-1
61+
files:
62+
- test-1-01.html
63+
```
64+
65+
#### Example 3: Ignore parent features
66+
67+
```yaml
68+
apply_mode: IGNORE_PARENT
69+
features:
70+
- name: reset-test-1
71+
files: "**"
72+
```
73+
74+
#### Example 4: Ignore parent features and listing out explicit files
75+
76+
```yaml
77+
apply_mode: IGNORE_PARENT
78+
features:
79+
- name: reset-test-2
80+
files:
81+
- reset-test-2-01.html
82+
- name: reset-test-1
83+
files:
84+
- reset-test-2-01.html
85+
```
86+
87+
### Evidence for the above examples:
88+
- Example 2 is necessary for directories such as
89+
https://wpt.fyi/results/css/selectors which has tests for multiple features.
90+
- Examples 3 and 4 are necessary to support marking up all of
91+
https://wpt.fyi/results/css/css-grid *except* the masonry/ and subgrid/
92+
subdirectories as being part of the grid feature.
93+
94+
### Schema
95+
96+
```json
97+
{
98+
"$schema": "http://json-schema.org/draft-06/schema#",
99+
"$ref": "#/definitions/Main",
100+
"definitions": {
101+
"Main": {
102+
"type": "object",
103+
"additionalProperties": false,
104+
"properties": {
105+
"features": {
106+
"type": "array",
107+
"description": "List of features",
108+
"items": {
109+
"$ref": "#/definitions/FeatureEntry"
110+
}
111+
},
112+
"apply_mode": {
113+
"type": "string",
114+
"anyOf": [
115+
{
116+
"const": "IGNORE_PARENT",
117+
"description": "Ignores features from previous parent directories."
118+
}
119+
]
120+
}
121+
},
122+
"required": [
123+
"features"
124+
],
125+
"title": "Main"
126+
},
127+
"FeatureEntry": {
128+
"type": "object",
129+
"additionalProperties": false,
130+
"properties": {
131+
"name": {
132+
"type": "string",
133+
"description": "The web feature key"
134+
},
135+
"files": {
136+
"oneOf": [
137+
{
138+
"const": "**",
139+
"description": "All files recursively",
140+
"title": "Files"
141+
},
142+
{
143+
"type": "array",
144+
"items": {
145+
"type": "string",
146+
"description": "A specific file within the current directory"
147+
}
148+
}
149+
150+
]
151+
}
152+
},
153+
"required": [
154+
"name",
155+
"files"
156+
],
157+
"title": "FeatureEntry"
158+
}
159+
}
160+
}
161+
```
162+
163+
## Step 2. Add linting functionality
164+
165+
There exists a potential risk of putting all the information in WEB_FEATURES.yml:
166+
the overrides which are linked to the file can go out of sync as test files are moved,
167+
renamed, or deleted.
168+
169+
As a result, this RFC also proposes a lint command to detect this. The code for
170+
this will reside in `tools/lint/lint.py`.
171+
172+
## Step 3. Create a script to generate a manifest
173+
174+
A command will be created in the `tools/manifest` folder to generate the
175+
WEB_FEATURES_MANIFEST.json file. While it is in that folder, its logic will
176+
be independent of the existing manifest logic. (As discussed in the August 01,
177+
2023 Monthly WPT
178+
[Meeting](https://github.com/web-platform-tests/wpt-notes/blob/master/minutes/2023-08-01.md#rfc-157---add-feature-meta-tag-to-web-platform-tests-meta-data),
179+
it should not use the same logic.) The only thing it will it
180+
do is generate data in the same structure.
181+
182+
By using the same format, wpt.fyi can use it to allow filtering by web feature.
183+
184+
The rules of the generation are described in the file schema above by the
185+
directory level `apply_mode` flag.
186+
187+
## Step 4. Adjust the wpt-pr-bot to handle reviews of the changes
188+
189+
### Current State of wpt-pr-bot
190+
191+
Currently, the wpt-pr-bot builds a list of PR reviewers by:
192+
1. Retrieving the paths for all of the files
193+
2. Finding the nearest META.yml file for each path
194+
3. Add the list of suggested_reviewers from a given META.yml to a set of
195+
reviewers
196+
197+
### Proposed changes
198+
199+
Have the wpt-pr-bot filter the WEB_FEATURES.yml file changes to only request
200+
reviews from web-features contributors.
201+
202+
This will reduce the amount of unneeded reviews from non web-features contributors.
203+
204+
---
205+
206+
# Risks
207+
208+
A set of new linting cases (mentioned in Step 2.) could confuse existing
209+
developers.
210+
211+
## Mitigation
212+
213+
Ways to mitigate this include:
214+
- Informative logging
215+
- Additional documentation to describe why this linting exists.
216+
217+
# Other Notes
218+
219+
## Changes for WPT Contributors
220+
221+
The metadata tag is not mandatory. WPT contributions will not be blocked if
222+
contributors do not add the metadata tag to the WEB_FEATURES.yml files or test files.
223+
224+
## Populating and maintaining the metadata files
225+
226+
As the [web-features](https://github.com/web-platform-dx/web-features) repository
227+
is populated with web-features definitions, web-features contributors will begin
228+
populating the metadata in WPT.
229+
230+
In regards to maintaining the metadata files, web-features contributors will be
231+
responsible for the mitigation options
232+
described above.
233+
# Roll back of this RFC
234+
235+
The following steps will allow the community to roll back this RFC in the event it is deemed unnecessary:
236+
237+
1. Remove all the new metadata files
238+
- ```sh
239+
find . -name WEB_FEATURES.yml -type f -delete
240+
```
241+
2. Remove the new web_features.py and reference in commands.json
242+
- This will likely happen by reverting the PRs in the tools/manifest folder.
243+
3. Remove the lint code in tools/lint/lint.py.
244+
- This will likely happen by reverting the PRs in the tools/lint/lint.py file.

0 commit comments

Comments
 (0)