Skip to content

Commit e12d44c

Browse files
YUNQIUGUOrachguorachguo
authored
Integrate React Native E2E test with detox framework (microsoft#15133)
### Description <!-- Describe your changes. --> Integrate react native e2e test framework with detox. https://wix.github.io/Detox/ Good build in CI: https://dev.azure.com/onnxruntime/onnxruntime/_build/results?buildId=946695&view=results ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. --> Write cross-platform end-to-end tests in JavaScript. Resolve flaky e2e tests in react native ci pipelines. --------- Co-authored-by: rachguo <[email protected]> Co-authored-by: rachguo <[email protected]>
1 parent a9bfeb9 commit e12d44c

File tree

21 files changed

+2250
-497
lines changed

21 files changed

+2250
-497
lines changed

README.md

Lines changed: 61 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -397,27 +397,78 @@ From ORT v1.13 onwards the 'full' ONNX Runtime package is used. It supports both
397397
- replace `com.microsoft.onnxruntime:onnxruntime-android` with `com.microsoft.onnxruntime:onnxruntime-mobile` in /js/react_native/e2e/android/app/build.gradle
398398
- replace `onnxruntime-c` with `onnxruntime-mobile-c` in /js/react_native/e2e/ios/Podfile
399399

400-
From `<ORT_ROOT>/js/react_native/e2e/android`, run e2e Android tests as follows,
400+
- Run E2E Testing with Detox framework
401401

402-
```sh
403-
./gradlew :app:connectedDebugAndroidTest
404-
```
402+
When testing with integrated [Detox](https://wix.github.io/Detox/docs/next/introduction/getting-started) framework for Android and iOS e2e apps:
403+
- Detox prerequisites:
405404

406-
From `<ORT_ROOT>/js/react_native/e2e/ios`, run e2e iOS tests as follows,
405+
Install detox command line tools:
406+
```
407+
yarn global add detox-cli
408+
```
409+
Install applesimutils which is required by Detox to work with iOS simulators. (Requires a MacOS device)
410+
```
411+
brew tap wix/brew
412+
brew install applesimutils
413+
```
414+
Main Detox project files:
415+
- `.detoxrc.js` -Detox config file;
416+
- `e2e/jest.config.js` -Jest configuration;
417+
- `e2e/OnnxruntimeModuleExample.test.js` - initial react native onnxruntimemodule e2e detox test.
418+
- Build the detox e2e testing app.
407419
408-
```sh
409-
xcrun xcodebuild test -workspace OnnxruntimeModuleExample.xcworkspace -scheme OnnxruntimeModuleExample -destination 'platform=iOS Simulator,OS=latest,name=iPhone 13'
410-
```
420+
From `<ORT_ROOT>/js/react_native/e2e`, run the command to build the e2e testing app. Before that ensure you have android emulator/ios simulator started locally.
421+
422+
iOS (Debug):
423+
424+
```
425+
detox build --configuration ios.sim.debug
426+
```
427+
428+
Android (Debug):
429+
430+
```
431+
detox build --configuration android.emu.debug
432+
```
433+
434+
* Note: If names of local testing android/ios devices do not match the default setting in `.detoxrc.js` file,
435+
modify the device name in config files accordingly to match local device name otherwise would cause a build failure.
411436
412-
***`yarn bootstrap` changes `packages.json` and `yarn.lock` files. Once testing is done, restore changes to avoid unwanted commit.***
437+
- Run the detox e2e tests.
438+
439+
In a debug configuration, you need to have React Native packager running in parallel before you start Detox tests:
440+
441+
```
442+
npm start
443+
444+
> react-native start
445+
```
446+
447+
From `<ORT_ROOT>/js/react_native/e2e`, run Detox tests using the following command:
448+
449+
iOS (Debug):
450+
451+
```
452+
detox test --configuration ios.sim.debug
453+
```
454+
455+
Android (Debug):
456+
457+
```
458+
detox test --configuration android.emu.debug
459+
```
460+
461+
To record logs for testing results, add `--record-logs`. Output logs and test results will be produced in the `e2e/artifacts/` folder.
462+
See: [Detox/logger#artifacts](https://wix.github.io/Detox/docs/api/logger#artifacts)
463+
464+
***`yarn bootstrap` changes `packages.json` and `yarn.lock` files. Once testing is done, restore changes to avoid unwanted commit.***
413465
414466
5. Run Android and iOS apps.
415467
416468
```sh
417469
yarn e2e android
418470
yarn e2e ios
419471
```
420-
421472
### NPM Packaging
422473

423474
1. Update a version using `npm version <version>` from `<ORT_ROOT>/js/react_native` folder. If it's for a dev, use `npm version <version>-dev.<subversion>`

react_native/e2e/.detoxrc.js

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
/** @type {Detox.DetoxConfig} */
2+
module.exports = {
3+
testRunner: {
4+
args: {
5+
'$0': 'jest',
6+
config: 'test/jest.config.js'
7+
},
8+
jest: {
9+
setupTimeout: 120000
10+
}
11+
},
12+
apps: {
13+
'ios.debug': {
14+
type: 'ios.app',
15+
binaryPath: 'ios/build/Build/Products/Debug-iphonesimulator/OnnxruntimeModuleExample.app',
16+
build: 'xcodebuild -workspace ios/OnnxruntimeModuleExample.xcworkspace -scheme OnnxruntimeModuleExample -configuration Debug -sdk iphonesimulator -derivedDataPath ios/build'
17+
},
18+
'ios.release': {
19+
type: 'ios.app',
20+
binaryPath: 'ios/build/Build/Products/Release-iphonesimulator/OnnxruntimeModuleExample.app',
21+
build: 'xcodebuild -workspace ios/OnnxruntimeModuleExample.xcworkspace -scheme OnnxruntimeModuleExample -configuration Release -sdk iphonesimulator -derivedDataPath ios/build'
22+
},
23+
'android.debug': {
24+
type: 'android.apk',
25+
binaryPath: 'android/app/build/outputs/apk/debug/app-debug.apk',
26+
build: 'cd android && ./gradlew assembleDebug assembleAndroidTest -DtestBuildType=debug',
27+
reversePorts: [
28+
8081
29+
]
30+
},
31+
'android.release': {
32+
type: 'android.apk',
33+
binaryPath: 'android/app/build/outputs/apk/release/app-release.apk',
34+
build: 'cd android && ./gradlew assembleRelease assembleAndroidTest -DtestBuildType=release'
35+
}
36+
},
37+
devices: {
38+
simulator: {
39+
type: 'ios.simulator',
40+
device: {
41+
type: 'iPhone 13'
42+
}
43+
},
44+
attached: {
45+
type: 'android.attached',
46+
device: {
47+
adbName: '.*'
48+
}
49+
},
50+
emulator: {
51+
type: 'android.emulator',
52+
device: {
53+
avdName: 'ort_android'
54+
}
55+
}
56+
},
57+
configurations: {
58+
'ios.sim.debug': {
59+
device: 'simulator',
60+
app: 'ios.debug'
61+
},
62+
'ios.sim.release': {
63+
device: 'simulator',
64+
app: 'ios.release'
65+
},
66+
'android.att.debug': {
67+
device: 'attached',
68+
app: 'android.debug'
69+
},
70+
'android.att.release': {
71+
device: 'attached',
72+
app: 'android.release'
73+
},
74+
'android.emu.debug': {
75+
device: 'emulator',
76+
app: 'android.debug'
77+
},
78+
'android.emu.release': {
79+
device: 'emulator',
80+
app: 'android.release'
81+
}
82+
}
83+
};

react_native/e2e/android/app/build.gradle

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -133,7 +133,7 @@ android {
133133
targetSdkVersion rootProject.ext.targetSdkVersion
134134
versionCode 1
135135
versionName "1.0"
136-
136+
testBuildType System.getProperty('testBuildType', 'debug')
137137
testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
138138
}
139139
splits {
@@ -159,6 +159,7 @@ android {
159159
signingConfig signingConfigs.debug
160160
minifyEnabled enableProguardInReleaseBuilds
161161
proguardFiles getDefaultProguardFile("proguard-android.txt"), "proguard-rules.pro"
162+
proguardFile "${rootProject.projectDir}/../node_modules/detox/android/detox/proguard-rules-app.pro"
162163
}
163164
}
164165
// applicationVariants are e.g. debug, release
@@ -184,6 +185,9 @@ repositories {
184185
}
185186

186187
dependencies {
188+
androidTestImplementation('com.wix:detox:20.7.0')
189+
implementation 'androidx.appcompat:appcompat:1.1.0'
190+
187191
implementation fileTree(dir: "libs", include: ["*.jar"])
188192
//noinspection GradleDynamicVersion
189193
implementation "com.facebook.react:react-native:+" // From node_modules
Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
// Dummy auxiliary Android Test
2+
// Detox requires the project to have a single dummy native Android Test with some special content,
3+
// which will be picked up by the testRunner.
4+
5+
package com.example.reactnativeonnxruntimemodule;
6+
7+
import com.wix.detox.Detox;
8+
import com.wix.detox.config.DetoxConfig;
9+
10+
import org.junit.Rule;
11+
import org.junit.Test;
12+
import org.junit.runner.RunWith;
13+
14+
import androidx.test.ext.junit.runners.AndroidJUnit4;
15+
import androidx.test.filters.LargeTest;
16+
import androidx.test.rule.ActivityTestRule;
17+
18+
@RunWith(AndroidJUnit4.class)
19+
@LargeTest
20+
public class DetoxTest {
21+
@Rule
22+
public ActivityTestRule<MainActivity> mActivityRule = new ActivityTestRule<>(MainActivity.class, false, false);
23+
24+
@Test
25+
public void runDetoxTests() {
26+
DetoxConfig detoxConfig = new DetoxConfig();
27+
detoxConfig.idlePolicyConfig.masterTimeoutSec = 90;
28+
detoxConfig.idlePolicyConfig.idleResourceTimeoutSec = 60;
29+
detoxConfig.rnContextLoadTimeoutSec = (BuildConfig.DEBUG ? 180 : 60);
30+
31+
Detox.runTests(mActivityRule, detoxConfig);
32+
}
33+
}

react_native/e2e/android/app/src/androidTest/java/com/example/reactnativeonnxruntimemodule/OnnxruntimeModuleExampleUITests.java

Lines changed: 0 additions & 119 deletions
This file was deleted.

react_native/e2e/android/app/src/main/AndroidManifest.xml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,8 @@
1111
android:icon="@mipmap/ic_launcher"
1212
android:roundIcon="@mipmap/ic_launcher_round"
1313
android:allowBackup="false"
14-
android:theme="@style/AppTheme">
14+
android:theme="@style/AppTheme"
15+
android:networkSecurityConfig="@xml/network_security_config">
1516
<activity
1617
android:name=".MainActivity"
1718
android:label="@string/app_name"
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
<?xml version="1.0" encoding="utf-8"?>
2+
<network-security-config>
3+
<domain-config cleartextTrafficPermitted="true">
4+
<domain includeSubdomains="true">10.0.2.2</domain>
5+
<domain includeSubdomains="true">localhost</domain>
6+
</domain-config>
7+
</network-security-config>

0 commit comments

Comments
 (0)