Skip to content

Commit 9a3dced

Browse files
committed
Add readme for flakeguard
1 parent aa22cd4 commit 9a3dced

File tree

1 file changed

+60
-0
lines changed

1 file changed

+60
-0
lines changed

tools/flakeguard/README.md

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
# Flakeguard
2+
3+
**Flakeguard** is a tool designed to help identify flaky tests within a Go project. Flaky tests are tests that intermittently fail without changes to the code, often due to race conditions or other non-deterministic behavior. Flakeguard assists by analyzing the impact of code changes on test packages and by running tests multiple times to determine stability.
4+
5+
## Features
6+
7+
- **Identify Impacted Tests**: Detects test packages that may be affected by changes in your Go project files.
8+
- **Run Tests for Flakiness**: Runs tests multiple times to determine their flakiness.
9+
- **Output Results in JSON**: Allows easy integration with CI pipelines and custom reporting.
10+
- **Supports Exclusion Lists**: Configurable to exclude specified packages or paths from the analysis.
11+
- **Recursive Dependency Analysis**: Detects all impacted packages through dependency levels.
12+
13+
## Installation
14+
15+
To install `flakeguard` CLI, you need to have Go installed on your machine. With Go installed, run the following command:
16+
17+
```sh
18+
go install github.com/smartcontractkit/chainlink-testing-framework/tools/flakeguard@latest
19+
```
20+
21+
## Usage
22+
23+
Flakeguard offers two main commands: `find` and `run`.
24+
25+
### `find` Command
26+
27+
The `find` command identifies test packages affected by recent changes.
28+
29+
#### Flags for `find` Command
30+
31+
- `--project-path, -r` : Path to the Go project. Defaults to the current directory.
32+
- `--base-ref` (required): Git reference (branch, tag, or commit) for comparing changes.
33+
- `--verbose, -v` : Enables verbose mode.
34+
- `--json` : Outputs results in JSON format.
35+
- `--filter-empty-tests` : Filters out test packages with no actual test functions (can be slow on large projects).
36+
- `--excludes` : List of paths to exclude.
37+
- `--levels, -l` : Levels of recursion for dependency search, with `0` for unlimited. `2` by default.
38+
- `--find-by-test-files-diff` : Enable mode to find affected test packages by changes in test files.
39+
- `--find-by-affected-packages` : Enable mode to find affected test packages based on changes in any project package.
40+
- `--only-show-changed-test-files` : Display only changed test files and exit.
41+
42+
### `run` Command
43+
44+
The `run` command executes tests multiple times to identify flaky tests.
45+
46+
#### Flags for `run` Command
47+
48+
- `--project-path, -r` : Path to the Go project. Defaults to the current directory.
49+
- `--test-packages-json` : JSON-encoded string of test packages to run.
50+
- `--test-packages` : List of test packages to run.
51+
- `--run-count, -c` : Number of times to run the tests.
52+
- `--race` : Enable race condition detection.
53+
- `--output-json` : File path to output test results in JSON format.
54+
- `--threshold` : Threshold (0-1) for determining flakiness (e.g., `0.8` to pass if 80% successful).
55+
- `--skip-tests` : List of tests to skip.
56+
- `--fail-fast` : Stop on the first failure if threshold is set to `1.0`.
57+
58+
### JSON Output
59+
60+
Both `find` and `run` commands support JSON output `--json`, making it easy to integrate Flakeguard with CI/CD pipelines and reporting tools.

0 commit comments

Comments
 (0)