Skip to content

Add Baseline regression notes #3187

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open

Add Baseline regression notes #3187

wants to merge 8 commits into from

Conversation

ddbeck
Copy link
Collaborator

@ddbeck ddbeck commented Jul 28, 2025

This is a work in progress toward fixing #1971.

This structures the existing Baseline regression comments as notes. I'm looking for early feedback on whether this structure for notes would be useful.

@github-actions github-actions bot added the feature definition Creating or defining new features or groups of features. label Jul 28, 2025
ddbeck
ddbeck commented Jul 28, 2025
View reviewed changes
features/streams.yml
Comment on lines -5 to -10
# TODO: https://github.com/web-platform-dx/web-features/issues/1971
# Status changed: https://github.com/web-platform-dx/web-features/pull/2358, https://github.com/web-platform-dx/web-features/pull/2491
# 2024-12-19 — low → false — Regressed status to match Caniuse, which considers support beginning at BYOB shipping.
# 2025-01-30 — false → high — Split BYOB into a separate "readable-byte-streams" feature. Linked that one to Caniuse.
# References:
# - https://caniuse.com/streams
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fussed over this for a while and ended up deciding to omit the note entirely. The first note would say that the feature regressed, the second note would be some new category (advance? unregression?) showing that we more or less reverted the second note. It seemed to me that the correct course of action in that scenario would be to withdraw the note, so that's what I've done.

@ddbeck ddbeck added package:web-features schema Schema changes, proposals, and bugs minor version required This PR requires a minor version semver release (vX.Y+1.0) labels Jul 28, 2025
rviscomi
rviscomi reviewed Jul 28, 2025
View reviewed changes
@github-actions github-actions bot added the documentation Improvements or additions to documentation label Aug 30, 2025
ddbeck
ddbeck commented Aug 30, 2025
View reviewed changes
features/popover.yml
Comment on lines +5 to +14
# notes:
# - date: 2024-09-18
# category: baseline-regression
# old_baseline_value: low
# new_baseline_value: false
# message: >
# Safari on iOS has a bug that prevents light dismiss (tapping outside the element to close it).
# citations:
# - https://bugs.webkit.org/show_bug.cgi?id=267688
# - https://github.com/mdn/browser-compat-data/issues/22927
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I opened this PR, I had written this note. However, the feature has since progressed. It occurred to me that we should not show irrelevant notes, so I commented it out. This is a rather new idea, so I'd like to delete this entirely, if we decide this is the right way to go.

Other options considered: some sort of "historic" flag on notes that are no longer relevant, or filtering historic notes from the published package. I figured both of those added complexity that wouldn't be very useful to anyone, least of all developers.

@ddbeck ddbeck marked this pull request as ready for review August 30, 2025 18:30
@ddbeck
Copy link
Collaborator Author

ddbeck commented Aug 30, 2025

I've updated this PR. Here's what's new:

  • The schema has been added. To match the description field, there is now a plain-text message field and a formatted message_html in the built package.

  • Regression notes that apply to a historic status are now forbidden. For example, if a feature regresses from high to low, then advances back to high, then the script that ingests the YAML treats that as an error. My working assumption is that such historic notes have no utility to web developers and ought to be dropped. This would result in the loss of one regression note. See my self-review for more detail and possible alternatives.

  • There are now guidelines for adding regression notes. I wrote the guidelines to emphasize impact on developers. This led to…

  • The regression notes have been revised. I tried to rewrite them to help developers make a decision about whether the regression is meaningful to their applications. This makes for much stronger notes, in my opinion.

  • The script that ingests the YAML files has been updated to handle the new fields and do the requisite conversions.

This PR now passes the tests and is ready for (real) review. I'll share this with the WebDX chat on Monday.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
1 more reviewer

@rviscomi rviscomi rviscomi left review comments

Reviewers whose approvals may not affect merge requirements

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
documentation Improvements or additions to documentation feature definition Creating or defining new features or groups of features. minor version required This PR requires a minor version semver release (vX.Y+1.0) package:web-features schema Schema changes, proposals, and bugs
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ddbeck @rviscomi