Skip to content

Commit 6aa3d10

Browse files
brageh01brageh01
authored
Merge branch 'main' into merge-and-extract-calendar-number-and-wrong-type-tests
2 parents d93849b + 61fcd7b commit 6aa3d10

File tree

12 files changed

+653
-456
lines changed

12 files changed

+653
-456
lines changed

‎CONTRIBUTING.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,10 @@
11
# Test262 Authoring Guidelines
22

3+
## So You Want To Write Tests For A Stage 2.7 Proposal
4+
5+
If you are writing tests for a whole proposal (not just for a few lines in the spec) please read about [testing plans](docs/testing-plan-guide.md).
6+
A **testing plan** showing what is and isn't intended to be covered, and **several small pull requests** implementing it in easily reviewable chunks is much preferred above a single pull request with all the tests.
7+
38
## Source Material
49

510
- https://tc39.github.io/ecma262/

‎docs/testing-plan-guide.md‎

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
# Testing plan guide
2+
3+
#### What is this document
4+
5+
[ECMAScript proposals](https://github.com/tc39/proposals/) that reach Stage 2.7 in the [TC39 Process](https://tc39.es/process-document/) need to have sufficient tests integrated to the Test262 test suite. To track the added tests, ensure that they are complete, and to get an idea of the amount of work needed, it's often helpful to have a testing plan outlining what exactly should be tested.
6+
7+
This document is a guide intended to help proposal authors and test262 contributors create such testing plans. It includes detailed instructions for contributors less familiar with test262 testing or less familiar with composing a testing plan, as well as a [checklist](./#testing-plan-checklist) that may also be helpful to contributors more experienced with Test262 tests and testing plans.
8+
9+
In the end of this document, in addition to the [checklist](./#testing-plan-checklist), there are also some [links](./#see-also) with further material.
10+
11+
12+
## Getting started creating a testing plan for an ECMAScript proposal.
13+
14+
If you are looking for an ECMAScript proposal to write a testing plan for, we try to have [open issues asking for testing plans](https://github.com/tc39/test262/issues?q=is%3Aissue+is%3Aopen+testing+plan), for proposals in Stage 2.7 or 3. You can also see the [TC39 document](https://github.com/tc39/proposals/) tracking the status of all the open proposals, in case we missed something, in which case please create the missing testing plan issue.
15+
16+
Before you start it's wise to communicate with proposal champion(s) in case they already have a testing plan and/or tests ready.
17+
18+
Understand the proposal text:
19+
20+
- read the explainer in the proposal repo's README, and
21+
- read the slides from any TC39 plenary presentation.
22+
23+
If you're unfamiliar with ECMAScript spec text, read this guide on [how to read the ECMAScript Specification](https://timothygu.me/es-howto/).
24+
25+
Read the proposal's rendered specification.
26+
27+
Finally, list all the separate tests that need to be written, as explained in the following sections.
28+
29+
### Tests for new APIs
30+
31+
#### Boilerplate tests
32+
33+
For new APIs, you can start by listing some standard tests, such as the following.
34+
35+
- Property descriptor of the object/method/whatever. [Example](https://github.com/tc39/test262/blob/main/test/built-ins/String/prototype/at/prop-desc.js)
36+
- Brand check for methods, what happens when calling it with a `this` that is not what what it expects, via `Function.prototype.apply` or `.call`. [Example](https://github.com/tc39/test262/blob/main/test/intl402/ListFormat/prototype/format/branding.js)
37+
- Object prototype and extensibility. [Example](https://github.com/tc39/test262/blob/main/test/built-ins/Temporal/Instant/builtin.js)
38+
- Function boilerplate:
39+
+ Function length. [Example](https://github.com/tc39/test262/blob/main/test/built-ins/String/prototype/at/length.js)
40+
+ Function name. [Example](https://github.com/tc39/test262/blob/main/test/built-ins/String/prototype/at/name.js)
41+
+ Function is or is not a constructor. [Example](https://github.com/tc39/test262/blob/main/test/built-ins/Number/isNaN/not-a-constructor.js)
42+
43+
#### Success cases
44+
45+
List the various success cases that result in a normal completion:
46+
47+
- Identify algorithm branches, and include a potential input that leads to each branch case.
48+
- See the proposal's explainer for particular cases that this proposal wants to cover.
49+
50+
#### Failure cases
51+
52+
List failure cases, when an exception is thrown:
53+
54+
- Check new spec text for every place where a `?` appears, indicating that what follows may throw an exception.
55+
- List other potential unexpected edge cases. At the rendered spec, use the `U` key to see who calls the related parts of the code, and check for unexpected entry points.
56+
57+
#### Side-effects and algorithm step order
58+
59+
Finally, consider testing the order of the algorithm steps, if there are steps that introduce side effects. However, it's best to not overdo it here. The goal is to help implementations correctly implement the new functionality. Exhaustively testing the order of side effects can lead to diminishing returns.
60+
61+
### Tests for new syntax
62+
63+
A testing plan for new syntax is similar to that for a new API but additionally:
64+
65+
- Test both the syntax and how it's evaluated, so double the tests.
66+
- Check [CONTRIBUTING.md](https://github.com/tc39/test262/blob/main/CONTRIBUTING.md) for special frontmatter flags when testing syntax errors.
67+
- Look into the ECMA-262 sections named [Runtime Semantics: Evaluation](https://tc39.es/ecma262/#sec-array-initializer-runtime-semantics-evaluation).
68+
69+
## Tips and tricks
70+
71+
Check what engines test, in case you missed something. Some engines' test locations in their codebase:
72+
73+
- v8 https://source.chromium.org/chromium/chromium/src/+/main:v8/test/mjsunit/
74+
- SpiderMonkey: https://searchfox.org/mozilla-central/source/js/src/tests/non262/ and https://searchfox.org/mozilla-central/source/js/src/jit-test/tests (note that the latter is usually more focused on testing the JIT compiler, however it may contain more general API tests that are not yet isolated as potential test262 tests).
75+
- JSC https://trac.webkit.org/browser/webkit/trunk/JSTests
76+
77+
Finally, if you don't understand why a certain part of the spec is the way it is, it helps to git blame the file `spec.html` of the proposal's Git repository. You may find linked PRs pointing to related issues, potentially with test cases that cover that part of the spec.
78+
79+
### Examples of input categories
80+
81+
- `undefined`, `null`, symbols
82+
- strings, "", "\b", "\n"
83+
- Booleans
84+
- number, BigInt, NaN, ±0, ±∞
85+
- Objects, Arrays
86+
87+
## Completing the testing plan
88+
89+
As important as it is to have a complete testing plan, it is equally important to avoid diminishing returns and to understand when to stop adding cases to be tested. Especially when new specification algorithms call on existing algorithms, it may be temping to dive in the existing algorithms to explore the different paths they may take. However it is best to focus on writing more surface tests for these cases, and only dive in the nested algorithms when the proposal is expecting these to be relevant to the new additions.
90+
91+
On this topic it may be helpful to go through the discussion on the [Testing Plan issue](https://github.com/tc39/test262/issues/4054) for `Math.sumPrecise`, and [Mike Pennisi's testing plan guide for WPT](https://web-platform-tests.org/writing-tests/making-a-testing-plan.html).
92+
93+
# Testing plan checklist
94+
95+
- [ ] Create a testing plan [issue](https://github.com/tc39/test262/issues), if there is none yet.
96+
+ [ ] Link the proposal repository and rendered text, if available.
97+
+ [ ] Optionally mention any implementations of the proposal, where the proposed tests could be tested.
98+
+ [ ] Cc the proposal champion(s) to inform them of the testing plan issue.
99+
- [ ] Send a PR to [tc39/proposals](https://github.com/tc39/proposals/) to link the testing plan issue ([example](https://github.com/tc39/proposals/pull/533)).
100+
- [ ] Compose a testing plan and add it to the issue as a checklist (using markdown checkbox list items `- [ ]`).
101+
- [ ] Cc the proposal champion(s) after the testing plan is added, so they can potentially weigh in on whether it is sufficient.
102+
103+
# See also
104+
105+
- Discussion on Testing Plan issue for `Math.sumPrecise`. https://github.com/tc39/test262/issues/4054
106+
- Mike Pennisi's guide for WPT. https://web-platform-tests.org/writing-tests/making-a-testing-plan.html
107+
- Other Test262 Testing Plan issues. https://github.com/tc39/test262/issues?q=is%3Aissue%20%22testing%20plan%22

‎test/built-ins/Array/fromAsync/sync-iterable-with-rejecting-thenable-closes.js‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
// Copyright (C) 2025 J. S. Choi. All rights reserved.
2+
// This code is governed by the BSD license found in the LICENSE file.
3+
4+
/*---
5+
esid: sec-array.fromasync
6+
description: >
7+
Array.fromAsync closes any sync-iterable input with a rejecting thenable.
8+
includes: [asyncHelpers.js]
9+
flags: [async]
10+
features: [Array.fromAsync]
11+
---*/
12+
13+
asyncTest(async function () {
14+
const expectedValue = {};
15+
let finallyCounter = 0;
16+
const inputThenable = {
17+
then(resolve, reject) {
18+
reject();
19+
},
20+
};
21+
function* createInput() {
22+
try {
23+
yield inputThenable;
24+
} finally {
25+
finallyCounter++;
26+
}
27+
}
28+
const input = createInput();
29+
try {
30+
await Array.fromAsync(input);
31+
} finally {
32+
assert.sameValue(finallyCounter, 1);
33+
}
34+
});

test/built-ins/Array/fromAsync/sync-iterable-with-thenable-element-rejects.js renamed to test/built-ins/Array/fromAsync/sync-iterable-with-rejecting-thenable-rejects.js

File renamed without changes.

‎test/intl402/DateTimeFormat/canonicalize-timezone.js‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,7 @@ info: |
5656
a. Let _timeZoneIdentifierRecord_ be GetAvailableNamedTimeZoneIdentifier(_timeZone_).
5757
b. If _timeZoneIdentifierRecord_ is ~empty~, throw a RangeError exception.
5858
c. Set _timeZone_ to _timeZoneIdentifierRecord_.[[PrimaryIdentifier]].
59+
features: [canonical-tz]
5960
---*/
6061

6162
const timeZones = [
@@ -93,10 +94,10 @@ const timeZones = [
9394
["Etc/GMT0", "UTC"]
9495
];
9596

96-
for (const [timeZone, expected] of timeZones) {
97+
for (const [timeZone, linkTarget] of timeZones) {
9798
assert.sameValue(
9899
new Intl.DateTimeFormat([], { timeZone }).resolvedOptions().timeZone,
99-
expected,
100-
"Time zone name " + timeZone + " should be canonicalized to " + expected
100+
timeZone,
101+
"Time zone name " + timeZone + " should be preserved and not canonicalized to " + linkTarget
101102
);
102103
}

‎test/intl402/DateTimeFormat/prototype/format/dayPeriod-long-en.js‎

Lines changed: 63 additions & 75 deletions
Original file line numberDiff line numberDiff line change
@@ -6,88 +6,76 @@ esid: sec-createdatetimeformat
66
description: Checks basic handling of dayPeriod, long format.
77
features: [Intl.DateTimeFormat-dayPeriod]
88
locale: [en]
9+
includes: [compareArray.js]
910
---*/
1011

11-
const d0000 = new Date(2017, 11, 12, 0, 0, 0, 0);
12-
const d0100 = new Date(2017, 11, 12, 1, 0, 0, 0);
13-
const d0200 = new Date(2017, 11, 12, 2, 0, 0, 0);
14-
const d0300 = new Date(2017, 11, 12, 3, 0, 0, 0);
15-
const d0400 = new Date(2017, 11, 12, 4, 0, 0, 0);
16-
const d0500 = new Date(2017, 11, 12, 5, 0, 0, 0);
17-
const d0600 = new Date(2017, 11, 12, 6, 0, 0, 0);
18-
const d0700 = new Date(2017, 11, 12, 7, 0, 0, 0);
19-
const d0800 = new Date(2017, 11, 12, 8, 0, 0, 0);
20-
const d0900 = new Date(2017, 11, 12, 9, 0, 0, 0);
21-
const d1000 = new Date(2017, 11, 12, 10, 0, 0, 0);
22-
const d1100 = new Date(2017, 11, 12, 11, 0, 0, 0);
23-
const d1200 = new Date(2017, 11, 12, 12, 0, 0, 0);
24-
const d1300 = new Date(2017, 11, 12, 13, 0, 0, 0);
25-
const d1400 = new Date(2017, 11, 12, 14, 0, 0, 0);
26-
const d1500 = new Date(2017, 11, 12, 15, 0, 0, 0);
27-
const d1600 = new Date(2017, 11, 12, 16, 0, 0, 0);
28-
const d1700 = new Date(2017, 11, 12, 17, 0, 0, 0);
29-
const d1800 = new Date(2017, 11, 12, 18, 0, 0, 0);
30-
const d1900 = new Date(2017, 11, 12, 19, 0, 0, 0);
31-
const d2000 = new Date(2017, 11, 12, 20, 0, 0, 0);
32-
const d2100 = new Date(2017, 11, 12, 21, 0, 0, 0);
33-
const d2200 = new Date(2017, 11, 12, 22, 0, 0, 0);
34-
const d2300 = new Date(2017, 11, 12, 23, 0, 0, 0);
12+
// Each expected dayPeriod value must be a) contiguous, and
13+
// b) represented in sequence.
14+
var expectedDayPeriods = [
15+
'in the morning',
16+
'noon',
17+
'in the afternoon',
18+
'in the evening',
19+
'at night'
20+
];
3521

36-
const long = new Intl.DateTimeFormat('en', {
22+
// Cover all 24 hours of a single day.
23+
var inputs = [];
24+
for (var h = 0; h < 24; h++) {
25+
inputs.push(new Date(2017, 11, 12, h, 0, 0, 0));
26+
}
27+
28+
// Verify complete and exclusive representation.
29+
var formatter = new Intl.DateTimeFormat('en', {
3730
dayPeriod: 'long'
3831
});
32+
var observedDayPeriods = [];
33+
var unexpectedDayPeriods = [];
34+
for (var h = 0; h < 24; h++) {
35+
var dayPeriod = formatter.format(inputs[h]);
36+
observedDayPeriods.push(dayPeriod);
37+
if (expectedDayPeriods.indexOf(dayPeriod) === -1) {
38+
unexpectedDayPeriods.push(dayPeriod);
39+
}
40+
}
41+
var unusedDayPeriods = expectedDayPeriods.filter(function (dayPeriod) {
42+
return observedDayPeriods.indexOf(dayPeriod) === -1;
43+
});
44+
assert.compareArray(unexpectedDayPeriods, [],
45+
'unexpected dayPeriods: ' + unexpectedDayPeriods.join());
46+
assert.compareArray(unusedDayPeriods, [],
47+
'unused dayPeriods: ' + unusedDayPeriods.join());
3948

40-
assert.sameValue(long.format(d0000), 'at night', '00:00, long format');
41-
assert.sameValue(long.format(d0100), 'at night', '01:00, long format');
42-
assert.sameValue(long.format(d0200), 'at night', '02:00, long format');
43-
assert.sameValue(long.format(d0300), 'at night', '03:00, long format');
44-
assert.sameValue(long.format(d0400), 'at night', '04:00, long format');
45-
assert.sameValue(long.format(d0500), 'at night', '05:00, long format');
46-
assert.sameValue(long.format(d0600), 'in the morning', '06:00, long format');
47-
assert.sameValue(long.format(d0700), 'in the morning', '07:00, long format');
48-
assert.sameValue(long.format(d0800), 'in the morning', '08:00, long format');
49-
assert.sameValue(long.format(d0900), 'in the morning', '09:00, long format');
50-
assert.sameValue(long.format(d1000), 'in the morning', '10:00, long format');
51-
assert.sameValue(long.format(d1100), 'in the morning', '11:00, long format');
52-
assert.sameValue(long.format(d1200), 'noon', '12:00, long format');
53-
assert.sameValue(long.format(d1300), 'in the afternoon', '13:00, long format');
54-
assert.sameValue(long.format(d1400), 'in the afternoon', '14:00, long format');
55-
assert.sameValue(long.format(d1500), 'in the afternoon', '15:00, long format');
56-
assert.sameValue(long.format(d1600), 'in the afternoon', '16:00, long format');
57-
assert.sameValue(long.format(d1700), 'in the afternoon', '17:00, long format');
58-
assert.sameValue(long.format(d1800), 'in the evening', '18:00, long format');
59-
assert.sameValue(long.format(d1900), 'in the evening', '19:00, long format');
60-
assert.sameValue(long.format(d2000), 'in the evening', '20:00, long format');
61-
assert.sameValue(long.format(d2100), 'at night', '21:00, long format');
62-
assert.sameValue(long.format(d2200), 'at night', '22:00, long format');
63-
assert.sameValue(long.format(d2300), 'at night', '23:00, long format');
49+
function arrayAt(arr, relIndex) {
50+
var realIndex = relIndex < 0 ? arr.length + relIndex : relIndex;
51+
if (realIndex < 0 || realIndex >= arr.length) return undefined;
52+
return arr[realIndex];
53+
}
6454

65-
const longNumeric = new Intl.DateTimeFormat('en', {
55+
// Verify ordering, accounting for the possibility of one value spanning day
56+
// transitions.
57+
var transitionCount = 0;
58+
for (var h = 0; h < 24; h++) {
59+
var dayPeriod = observedDayPeriods[h];
60+
var prevDayPeriod = arrayAt(observedDayPeriods, h - 1);
61+
if (dayPeriod === prevDayPeriod) continue;
62+
transitionCount++;
63+
var i = expectedDayPeriods.indexOf(dayPeriod);
64+
assert.sameValue(prevDayPeriod, arrayAt(expectedDayPeriods, i - 1),
65+
dayPeriod + ' must be preceded by ' + prevDayPeriod);
66+
}
67+
assert.sameValue(transitionCount, expectedDayPeriods.length,
68+
'dayPeriods must be contiguous');
69+
70+
var numericFormatter = new Intl.DateTimeFormat('en', {
6671
dayPeriod: 'long',
6772
hour: 'numeric'
6873
});
69-
70-
assert.sameValue(longNumeric.format(d0000), '12 at night', '00:00, long-numeric');
71-
assert.sameValue(longNumeric.format(d0100), '1 at night', '01:00, long-numeric');
72-
assert.sameValue(longNumeric.format(d0200), '2 at night', '02:00, long-numeric');
73-
assert.sameValue(longNumeric.format(d0300), '3 at night', '03:00, long-numeric');
74-
assert.sameValue(longNumeric.format(d0400), '4 at night', '04:00, long-numeric');
75-
assert.sameValue(longNumeric.format(d0500), '5 at night', '05:00, long-numeric');
76-
assert.sameValue(longNumeric.format(d0600), '6 in the morning', '06:00, long-numeric');
77-
assert.sameValue(longNumeric.format(d0700), '7 in the morning', '07:00, long-numeric');
78-
assert.sameValue(longNumeric.format(d0800), '8 in the morning', '08:00, long-numeric');
79-
assert.sameValue(longNumeric.format(d0900), '9 in the morning', '09:00, long-numeric');
80-
assert.sameValue(longNumeric.format(d1000), '10 in the morning', '10:00, long-numeric');
81-
assert.sameValue(longNumeric.format(d1100), '11 in the morning', '11:00, long-numeric');
82-
assert.sameValue(longNumeric.format(d1200), '12 noon', '12:00, long-numeric');
83-
assert.sameValue(longNumeric.format(d1300), '1 in the afternoon', '13:00, long-numeric');
84-
assert.sameValue(longNumeric.format(d1400), '2 in the afternoon', '14:00, long-numeric');
85-
assert.sameValue(longNumeric.format(d1500), '3 in the afternoon', '15:00, long-numeric');
86-
assert.sameValue(longNumeric.format(d1600), '4 in the afternoon', '16:00, long-numeric');
87-
assert.sameValue(longNumeric.format(d1700), '5 in the afternoon', '17:00, long-numeric');
88-
assert.sameValue(longNumeric.format(d1800), '6 in the evening', '18:00, long-numeric');
89-
assert.sameValue(longNumeric.format(d1900), '7 in the evening', '19:00, long-numeric');
90-
assert.sameValue(longNumeric.format(d2000), '8 in the evening', '20:00, long-numeric');
91-
assert.sameValue(longNumeric.format(d2100), '9 at night', '21:00, long-numeric');
92-
assert.sameValue(longNumeric.format(d2200), '10 at night', '22:00, long-numeric');
93-
assert.sameValue(longNumeric.format(d2300), '11 at night', '23:00, long-numeric');
74+
for (var h = 0; h < 24; h++) {
75+
assert.sameValue(
76+
numericFormatter.format(inputs[h]),
77+
// Hour "00" is represented as "12".
78+
((h % 12) || 12) + ' ' + observedDayPeriods[h],
79+
'numeric hour must precede dayPeriod'
80+
);
81+
}

0 commit comments

Comments
 (0)