Skip to content

Commit ce86749

Browse files
committed
feat(testing): Initial draft
1 parent 3e3ab52 commit ce86749

File tree

1 file changed

+73
-0
lines changed

1 file changed

+73
-0
lines changed

text/0000-t-test.md

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
- Feature Name: n/a
2+
- Start Date: (2023-06-07)
3+
- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000)
4+
- Rust Issue: n/a
5+
6+
# Summary
7+
[summary]: #summary
8+
9+
Create a team focused on the testing in the development workflow under primarily under [t-devtools](https://www.rust-lang.org/governance/teams/dev-tools) but also working closely with [t-libs](https://www.rust-lang.org/governance/teams/library) when it comes to the API and implementation of the built-in test harness.
10+
11+
# Motivation
12+
[motivation]: #motivation
13+
14+
In [rust 1.70.0](https://blog.rust-lang.org/2023/06/01/Rust-1.70.0.html), a bug was fixed so unstable test features, like `cargo test -- --format json`, require using a nightly toolchain, like other unstable features. The problem is IDEs and CIs have been relying on this behavior to get programmatic test data and little progress has been made in the last 5 years ([#49359](https://github.com/rust-lang/rust/issues/49359)).
15+
16+
Before this, there was a growing interest in improving testing generally, like [better CI integration](https://internals.rust-lang.org/t/pre-rfc-implementing-test-binary-list-format-json-for-use-by-ide-test-explorers-runners/18308).
17+
18+
One challenge with improving the situation is that a lot of concerns cross multiple teams, in particular cargo (for `cargo test`), libs (for libtest). This makes it more difficult to establish a vision and coordinate efforts to achieve that vision.
19+
20+
There is also precedence for this today with t-devtools having more specific sub-teams like rustdoc and rustfmt.
21+
22+
## Mission and responsibilities
23+
[mission]: #mission
24+
25+
This team would primarily be focused on iterating on the test writing experience, `cargo test`, and enabling integration with external tools like CI or IDEs.
26+
27+
Examples issues to resolve:
28+
- [Stabilize support for programmatic (json) test output](https://github.com/rust-lang/rust/issues/49359)
29+
- What parts of [cargo nextest](https://nexte.st/) can we stabilize?
30+
- With the proliferation of test frameworks (e.g. [rstest], [trybuild], [trycmd], [cargo_test_support], [criterion]), are there underlying needs we can resolve?
31+
32+
## Relationships to other teams
33+
34+
T-devtools: This will be the parent team.
35+
36+
**T-cargo**: This is a sibling team that T-testing will need to work with similarly to T-rustfmt, T-clippy, etc.
37+
38+
**T-rustdoc**: This is a sibling team that T-testing will likely coordinate with if any changes are need to how we do doctesting
39+
40+
**T-libs**: This will effectively be another parent team as they are ultimately responsible for libtest.
41+
42+
## Processes
43+
44+
For decisions on vision and direction, T-testing will use a standard FCP process. T-testing will be subject to [T-cargo's processes](https://doc.crates.io/contrib/team.html#decision-process) when dealing with `cargo test` and T-libs's processes for libtest. For any newly developed crates and libraries, we will follow [T-cargo's processes](https://doc.crates.io/contrib/team.html#decision-process).
45+
46+
## Membership
47+
48+
Team members are expected to shepherd testing discussions and vote on FCPs and is independent of regular code contributions though contributing can help build up the relevant experience and/or demonstrate the qualities for team membership.
49+
50+
Qualifications that will be taken into account for membership include:
51+
52+
- Does the person have the judgement for when deciding when a larger consensus is needed?
53+
- Does the person understand the constraints of backwards compatibility within `cargo test` and libtest and exercise caution when extending the compatibly constraints?
54+
55+
Someone can become a member of the Cargo Team by requesting a review or being nominated by one of the existing members. They can be added by unanimous consent of the team. The team lead or another member of the team will also confirm with the moderation team that there are no concerns involving the proposed team member.
56+
57+
Leads are responsible for scheduling and facilitating team meetings and will be selected from the team members by consensus.
58+
59+
The initial members of the testing team shall be:
60+
- Lead: Caleb Cartwright (@calebcartwright)
61+
- Ed Page (@epage)
62+
- Weihang Lo (@weihanglo)
63+
- Scott Schafer (@Muscraft)
64+
65+
## Drawbacks and Alternatives
66+
67+
- The ownership lines are becoming more muddled with the two teams dealing with `cargo test` and two teams dealing with libtest.
68+
69+
[rstest]: https://crates.io/crates/rstest
70+
[trybuild]: https://crates.io/crates/trybuild
71+
[trycmd]: https://crates.io/crates/trycmd
72+
[cargo_test_support]: https://doc.rust-lang.org/nightly/nightly-rustc/cargo_test_support/
73+
[criterion]: https://crates.io/crates/criterion

0 commit comments

Comments
 (0)