diff --git a/proposals/0501-swiftpm-html-coverage-report.md b/proposals/0501-swiftpm-html-coverage-report.md index c896e031b2..07e182f9d1 100644 --- a/proposals/0501-swiftpm-html-coverage-report.md +++ b/proposals/0501-swiftpm-html-coverage-report.md @@ -5,7 +5,7 @@ * Review Manager: [David Cummings](https://github.com/daveyc123) * Status: **Active Review (December 7 - December 22, 2025)** * Implementation: [swiftlang/swift-package-manager#9076](https://github.com/swiftlang/swift-package-manager/pull/9076) -* Review: +* Review: * [Pitch](https://forums.swift.org/t/pitch-adding-html-coverage-support/82358) * [Review](https://forums.swift.org/t/se-0501-html-coverage-report/83601/1) @@ -70,7 +70,6 @@ make use of LLVM's tools and construct the proper command line arguments to the The proposted command line changes are as follows: - ### Format Selection The `swift test` command line will have an option named `--coverage-format`, @@ -81,7 +80,7 @@ specified. The command line option will be similar to: ```sh - --codecov-format, --code-coverage-format, --coverage-format + --coverage-format Format of the code coverage output. Can be specified multiple times. (default: json) json - Produces a JSON coverage report. html - Produces an HTML report produced by llvm-cov. @@ -90,45 +89,47 @@ The command line option will be similar to: ### Coverage Report configuration -`llvm-cov show` has several report configurability options. In order to -prevent a "command line arguments" explosion to `swift test`, the configuration -options will be read from a response file. The optional response file will be -located in `/.swiftpm/configuration/coverage.html.report.args.txt`. The -response file will be supported. +`llvm-cov show` has several report configurability options. In order to allow +for HTML coverage report customization, and to prevent a "command linearguments" +explosion to `swift test`, a `-Xcov` command line option will be added. These +argument will be passed directly to the underlying `llvm-cov` executable in the +order they appear. + +Since multiple coverage report can be specified in a single `swift test` invocation, +we must provide a way to ensure which argument is passed to one format, and not the other. + +Take this example + +``` +swift test --enable-coverage --coverage-format html --coverage-format json -Xcov --title -Xcov "My title" +``` + +The `-Xcov` arguments are ony supported when generating the HTML report. The value of `-Xcov` will have the following syntax -The user can include `--format=text`, or a variation thereof, in the response -file. In order to ensure SwiftPM will always generate an HTML report, SwiftPM -will add `--format=html` after the response file argument to ensure `llvm-cov` -will generate an HTML report. +``` +-Xcov [=] +``` +Here are some examples: + +- `-Xcov html=--title`: argument `--title` is only sent to the HTML coverage report generation +- `-Xcov json=myarg`: argument `myarg` is only sent to the JSON coverage report +- `-Xcov commonArg`: the argument `commonArg` is sent to all coverage format reports. +- `-Xcov notASupportedFormat=value`: the argument `notASupportedFormat=value` is sent to all coverage format reports. -SwiftPM will not perform any validation on the response file contents, except -to determine the output location. ### Coverage report location By default, the HTML report will be created in location under the scratch path -(ie: the build directory). However, this can be overridden using the response file. +(ie: the build directory). However, this can be overridden using `-Xcov` argument. Some CI system, such as [Jenkins](https://www.jenkins.io), only allow archiving contents files/directories that belong in a "sandbox" location. It can be a safe assumption that the CI system will have a copy of the repository in the "sandbox" location, allowing this system to upload the HTML report. -``` - --show-codecov-path [mode], --show-code-coverage-path [mode], --show-coverage-path [mode] - Print the path of the exported code coverage files. The mode specifies how to - display the paths of the selected code coverage file formats. (default: text) - json - Display the output in JSON format. - text - Display the output as plain text. - --show-codecov-path-mode, --show-code-coverage-path-mode, --show-coverage-path-mode - --enable-codecov, --enable-code-coverage, --enable-coverage/--disable-codecov, --disable-code-coverage, --disable-coverage - Enable code coverage. (default: --disable-codecov) - --codecov-format, --code-coverage-format, --coverage-format - Format of the code coverage output. Can be specified multiple times. (values: json, html; default: Produces a JSON coverage report.) -``` - ### Show coverage path + Prior to this proposal `swift test --show-coverage-path` would display a single absolute path location to the JSON coverage report. @@ -144,6 +145,15 @@ The `--show-coverage-path` command line argument will be modified to be an optio with a default value. The supported values are `json` or `text`, with the default being `text` to preverse existing behaviour. +The help test of the `--show-coverage-path` default as flage optional is: +``` + --show-coverage-path [] + Print the path of the exported code coverage files. The mode specifies how to + display the paths of the selected code coverage file formats. (default: text) + json - Display the output in JSON format. + text - Display the output as plain text. +``` + A value of `json` will output a JSON object with the key representing the format, and the value representing the output location of said format. @@ -177,6 +187,50 @@ Build of product 'swift-test' complete! (0.40s) } ``` +### Consolidate coverage option to use same argument "style" + +Prior to this feature, there are 2 coverage options + +``` + --show-codecov-path, --show-code-coverage-path, --show-coverage-path + Print the path of the exported code coverage JSON + --enable-code-coverage/--disable-code-coverage + Enable code coverage. (default: + --disable-code-coverage) +``` + +There are 3 ways to show the coverage path. This proposal recommends to modify all +coverage command line option to be a single, more useful option + +``` + --show-coverage-path [] + Print the path of the exported code coverage files. The mode specifies how to + display the paths of the selected code coverage file formats. (default: text) + json - Display the output in JSON format. + text - Display the output as plain text. + --enable-coverage/--disable-coverage + Enable code coverage. (default: --disable-coverage) +``` + +This requires gracefully phasing out the previous option in favour of the new one. + +### Coverage command line options + +TODO: update this command line options. +``` +COVERAGE OPTIONS: + --show-coverage-path [] + Print the path of the exported code coverage files. The mode specifies how to + display the paths of the selected code coverage file formats. (default: text) + json - Display the output in JSON format. + text - Display the output as plain text. + --enable-coverage/--disable-coverage + Enable code coverage. (default: --disable-coverage) + --coverage-format + Format of the code coverage output. Can be specified multiple times. (values: json, html; default: Produces a JSON coverage report.) + -Xcov Pass flag through to the underlying coverage report tool. Syntax is `[=]` +``` + ## Security Since SwiftPM will use `llvm-cov show`, there may be security implications from @@ -191,17 +245,36 @@ No impact is expected. ## Alternatives considered -- In addition to the response file, the coverage report generation can support - a command line argument similar to `-Xlinker`, `-Xcc` and others, which will - pass the arguments to `llvm-cov show` and override the values in the response - file. This has _not_ been implemented in the [PR]. -- Instead of having a `--show-codecov-path` as a tri-state, we could preserve - `--show-codecov-path` original behaviour and add an additional command line - argument to indicate the output mode. The comand line argument would be - `--show-codecov-path-mode `, where `` is either `text` or `json`. - This was not favours as `--show-codecov-path-mode` would have a dependency on - `--show-codecov-path` argument, and may lead to some confusion. + +### `--show-coverage-path` alternative -[PR]: https://github.com/swiftlang/swift-package-manager/pull/9076 +Instead of having a `--show-coverage-path` as a tri-state, we could preserve +`--show-coverage-path` original behaviour and add an additional command line +argument to indicate the output mode. The comand line argument would be +`--show-coverage-path-mode `, where `` is either `text` or `json`. +This was not favoured as `--show-coverage-path-mode` would have a dependency on +`--show-coverage-path` argument, and may lead to some confusion.