Skip to content

fix(tester): correct edition mapping for ES16#5146

Open
Xavrir wants to merge 3 commits intoboa-dev:mainfrom
Xavrir:fix/tester-edition-5053
Open

fix(tester): correct edition mapping for ES16#5146
Xavrir wants to merge 3 commits intoboa-dev:mainfrom
Xavrir:fix/tester-edition-5053

Conversation

@Xavrir
Copy link
Contributor

@Xavrir Xavrir commented Mar 19, 2026

Summary

  • map arraybuffer-transfer to ES15 and set-methods to ES16 in the test262 feature table
  • add SpecEdition::ES16 support to tester stats and backward-compatible result deserialization
  • add focused tests for the edition mappings and ES16 stats handling

Testing

  • cargo test -p boa_tester
  • cargo clippy -p boa_tester --all-features --all-targets
  • pre-push cargo make run-ci

Closes #5053

@Xavrir Xavrir requested a review from a team as a code owner March 19, 2026 03:53
@github-actions github-actions bot added C-Tests Issues and PRs related to the tests. C-Builtins PRs and Issues related to builtins/intrinsics Waiting On Review Waiting on reviews from the maintainers labels Mar 19, 2026
@github-actions github-actions bot added this to the v1.0.0 milestone Mar 19, 2026
@github-actions
Copy link

github-actions bot commented Mar 19, 2026

Test262 conformance changes

Test result main count PR count difference
Total 52,963 52,963 0
Passed 50,073 50,073 0
Ignored 2,072 2,072 0
Failed 818 818 0
Panics 0 0 0
Conformance 94.54% 94.54% 0.00%

Tested main commit: 055ee0958ce332f3f99af27536a7c6f9a91def27
Tested PR commit: 50e871f0fcc614cb651ed22d2e202d87660d8f55
Compare commits: 055ee09...50e871f

Comment on lines +456 to +502
#[cfg(test)]
mod tests {
use super::SpecEdition;
use crate::{
Locale,
read::{MetaData, TestFlag},
};

fn metadata(features: &[&str]) -> MetaData {
MetaData {
description: "test".into(),
esid: None,
es5id: None,
es6id: None,
info: "".into(),
features: features
.iter()
.map(|feature| (*feature).into())
.collect::<Vec<Box<str>>>()
.into_boxed_slice(),
includes: Box::default(),
flags: Box::<[TestFlag]>::default(),
negative: None,
locale: Locale::default(),
}
}

#[test]
fn maps_arraybuffer_transfer_to_es15() {
let metadata = metadata(&["arraybuffer-transfer"]);

assert_eq!(
SpecEdition::from_test_metadata(&metadata),
Ok(SpecEdition::ES15)
);
}

#[test]
fn maps_set_methods_to_es16() {
let metadata = metadata(&["set-methods"]);

assert_eq!(
SpecEdition::from_test_metadata(&metadata),
Ok(SpecEdition::ES16)
);
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need tests for these. Not because tests are bad or anything, but because the editions mapping is so in-flight sometimes that it's not really useful to test if specific features map to a certain version.

Case in point, if we had tests for these, then you wouldn't have been able to change the edition of these two features without changing the tests.

Copy link
Contributor Author

@Xavrir Xavrir Mar 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, I removed those mapping tests in 50e871f. You’re right that pinning exact feature editions there would just create churn.

}

#[test]
fn apply_updates_es16_statistics() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test doesn't really reflect how we keep count of tests; usually if ES6 has a test, that test also counts for the total in ES7 and onwards, so it doesn't make sense to have ES15 with a lower test count than ES16

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the extra ES16 stats tests in 50e871f and kept the PR focused on the edition fix plus the required tester plumbing.

@codecov
Copy link

codecov bot commented Mar 19, 2026

Codecov Report

❌ Patch coverage is 0% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 59.50%. Comparing base (6ddc2b4) to head (50e871f).
⚠️ Report is 890 commits behind head on main.

Files with missing lines Patch % Lines
tests/tester/src/main.rs 0.00% 7 Missing ⚠️
tests/tester/src/edition.rs 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main    #5146       +/-   ##
===========================================
+ Coverage   47.24%   59.50%   +12.26%     
===========================================
  Files         476      580      +104     
  Lines       46892    63189    +16297     
===========================================
+ Hits        22154    37602    +15448     
- Misses      24738    25587      +849     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Xavrir Xavrir requested a review from jedel1043 March 19, 2026 11:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

C-Builtins PRs and Issues related to builtins/intrinsics C-Tests Issues and PRs related to the tests. Waiting On Review Waiting on reviews from the maintainers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

edition.rs lists wrong ES edition for arraybuffer-transfer / set-methods

2 participants