Skip to content

Commit 828404a

Browse files
Add support for uniform sharding on instrumentation tests
1 parent cdd9f62 commit 828404a

File tree

4 files changed

+23
-6
lines changed

4 files changed

+23
-6
lines changed

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,7 @@ You can also run this step directly with [Bitrise CLI](https://github.com/bitris
6767
| `inst_test_runner_class` | The fully-qualified Java class name of the instrumentation test runner (leave empty to use the last name extracted from the APK manifest). | | |
6868
| `inst_test_targets` | A list of one or more instrumentation test targets to be run (default: all targets). Each target must be fully qualified with the package name or class name, in one of these formats: - `package package_name` - `class package_name.class_name` - `class package_name.class_name#method_name` For example: `class com.my.company.app.MyTargetClass,class com.my.company.app.MyOtherTargetClass` | | |
6969
| `inst_use_orchestrator` | The option of whether running each test within its own invocation of instrumentation with Android Test Orchestrator or not. | required | `false` |
70+
| `inst_num_uniform_shards` | The number of uniform shards across which to distribute test cases. The shards are run in parallel on separate devices, a value between 1 and 50. | | |
7071
| `robo_initial_activity` | The initial activity used to start the app during a robo test. (leave empty to get it extracted from the APK manifest) | | |
7172
| `robo_max_depth` | The maximum depth of the traversal stack a robo test can explore. Needs to be at least 2 to make Robo explore the app beyond the first activity(leave empty to use the default value: `50`) | | |
7273
| `robo_max_steps` | The maximum number of steps/actions a robo test can execute(leave empty to use the default value: `no limit`). | | |

config.go

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -52,10 +52,11 @@ type ConfigsModel struct {
5252
VerboseLog bool `env:"use_verbose_log,opt[true,false]"`
5353

5454
// instrumentation
55-
InstTestPackageID string `env:"inst_test_package_id"`
56-
InstTestRunnerClass string `env:"inst_test_runner_class"`
57-
InstTestTargets string `env:"inst_test_targets"`
58-
UseOrchestrator bool `env:"inst_use_orchestrator,opt[true,false]"`
55+
InstTestPackageID string `env:"inst_test_package_id"`
56+
InstTestRunnerClass string `env:"inst_test_runner_class"`
57+
InstTestTargets string `env:"inst_test_targets"`
58+
UseOrchestrator bool `env:"inst_use_orchestrator,opt[true,false]"`
59+
NumberOfUniformShards int `env:"inst_num_uniform_shards,range[0..50]"`
5960

6061
// robo
6162
RoboInitialActivity string `env:"robo_initial_activity"`
@@ -113,6 +114,7 @@ func (configs *ConfigsModel) print() {
113114
log.Printf("- InstTestRunnerClass: %s", configs.InstTestRunnerClass)
114115
log.Printf("- InstTestTargets: %s", configs.InstTestTargets)
115116
log.Printf("- UseOrchestrator: %t", configs.UseOrchestrator)
117+
log.Printf("- NumberOfUniformShards: %d", configs.NumberOfUniformShards)
116118
}
117119

118120
//robo

step.yml

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
1-
title: "Virtual Device Testing for Android"
2-
summary: Run Android UI tests on virtual devices
1+
title: "Virtual Device Testing for Android (Shard)"
2+
summary: Run Android UI tests on virtual devices (Shard)
33
description: |-
44
Run Android UI tests on virtual devices. This Step collects the built APK/AAB file from the `$BITRISE_APK_PATH` and in case of instrumentation tests, the `$BITRISE_TEST_APK_PATH` Environment Variables and uses Firebase Test Lab to run UI tests on them.
55
@@ -202,6 +202,14 @@ inputs:
202202
value_options:
203203
- "false"
204204
- "true"
205+
- inst_num_uniform_shards:
206+
opts:
207+
title: "Uniform shards number"
208+
summary: |
209+
The number of shards across which to distribute test cases. The shards are run in parallel on separate devices.
210+
description: |
211+
A value between 1 and 50. For example, if your test execution contains 20 test cases and you specify four shards, the instrumentation command passes arguments of `-e numShards 4` to `AndroidJUnitRunner` and each shard executes about five test cases. Based on the sharding mechanism `AndroidJUnitRunner` uses, there is no guarantee that test cases will be distributed with perfect uniformity.
212+
If zero is specified then no uniform sharding is applied and all test cases run on a single device.
205213
- robo_initial_activity:
206214
opts:
207215
category: "Robo Test"

test_api.go

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -208,6 +208,12 @@ func startTestRun(configs ConfigsModel, testAssets TestAssetsAndroid) error {
208208
} else {
209209
testModel.TestSpecification.AndroidInstrumentationTest.OrchestratorOption = "DO_NOT_USE_ORCHESTRATOR"
210210
}
211+
212+
if int64(configs.NumberOfUniformShards) > 0 {
213+
testModel.TestSpecification.AndroidInstrumentationTest.ShardingOption = &testing.ShardingOption{}
214+
testModel.TestSpecification.AndroidInstrumentationTest.ShardingOption.UniformSharding = &testing.UniformSharding{}
215+
testModel.TestSpecification.AndroidInstrumentationTest.ShardingOption.UniformSharding.NumShards = int64(configs.NumberOfUniformShards)
216+
}
211217
log.Debugf("AndroidInstrumentationTest: %+v", testModel.TestSpecification.AndroidInstrumentationTest)
212218
case testTypeRobo:
213219
testModel.TestSpecification.AndroidRoboTest = &testing.AndroidRoboTest{}

0 commit comments

Comments
 (0)