Local driver test all passed but aws device farm Tests failed #117
Description
sylph.yaml
tmp_dir: /tmp/sylph
artifacts_dir: /tmp/sylph_artifacts
sylph_timeout: 720
concurrent_runs: true
# device farm config
project_name: App Integration Tests
default_job_timeout: 10 #Dakika
device_pools:
- pool_name: android pool 1
pool_type: android
devices:
- name: Google Pixel 3a
model: G020G
os: 10
test_suites:
- test_suite: example tests 1
main: test_driver/main.dart
tests:
- test_driver/main_test.dart
pool_names:
- android pool 1
job_timeout: 15 #Dakika
main_test.dart
import 'package:flutter_driver/flutter_driver.dart';
import 'package:test/test.dart';
void main() {
group("end-to-end test", () {
FlutterDriver driver;
setUpAll(() async {
driver = await FlutterDriver.connect();
});
tearDownAll(() async {
if (driver != null) driver.close();
});
test('tap on the floating action button; verify counter', () async {
SerializableFinder fab = find.byTooltip('Increment');
await driver.waitFor(fab);
await driver.tap(fab);
await driver.waitFor(find.text('1'));
});
});
}
Test spec output:
[DEVICEFARM] Setting up your device. This usually takes 2-3 minutes.
[DEVICEFARM] ########### Start executing testspec ###########
[DEVICEFARM] ########### Entering phase install ###########
[DeviceFarm] cd flutter_app
[DeviceFarm] ls -la
total 88
drwxrwxrwx 10 rwx rwx 4096 Nov 23 07:12 .
drwxrwxrwx 9 rwx rwx 4096 Nov 23 07:12 ..
drwxrwxrwx 5 rwx rwx 4096 Nov 23 07:12 android
-rwxrwxrwx 1 rwx rwx 47 Nov 23 07:12 build_to_os.txt
drwxrwxrwx 3 rwx rwx 4096 Nov 23 07:12 .dart_tool
-rwxrwxrwx 1 rwx rwx 778 Nov 23 07:12 .gitignore
drwxrwxrwx 4 rwx rwx 4096 Nov 23 07:12 .idea
drwxrwxrwx 6 rwx rwx 4096 Nov 23 07:12 ios
drwxrwxrwx 2 rwx rwx 4096 Nov 23 07:12 lib
-rwxrwxrwx 1 rwx rwx 315 Nov 23 07:12 .metadata
-rwxrwxrwx 1 rwx rwx 6223 Nov 23 07:12 .packages
-rwxrwxrwx 1 rwx rwx 10549 Nov 23 07:12 pubspec.lock
-rwxrwxrwx 1 rwx rwx 371 Nov 23 07:12 pubspec.yaml
-rwxrwxrwx 1 rwx rwx 570 Nov 23 07:12 README.md
drwxrwxrwx 2 rwx rwx 4096 Nov 23 07:12 script
-rwxrwxrwx 1 rwx rwx 913 Nov 23 07:12 sylph_aws_farm_test1.iml
-rwxrwxrwx 1 rwx rwx 528 Nov 23 07:12 sylph.yaml
drwxrwxrwx 2 rwx rwx 4096 Nov 23 07:12 test
drwxrwxrwx 2 rwx rwx 4096 Nov 23 07:12 test_driver
[DeviceFarm] bash --version
GNU bash, version 4.3.11(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
[DeviceFarm] echo "DEVICEFARM_APP_PATH=$DEVICEFARM_APP_PATH"
DEVICEFARM_APP_PATH=/tmp/scratcheLzkfj.scratch/share-uIEcJ3.scratch/app-WG_wXw.apk
[DeviceFarm] echo "Install flutter"
Install flutter
[DeviceFarm] FLUTTER_VERSION=v1.12.13+hotfix.8-stable
[DeviceFarm] if [ $DEVICEFARM_DEVICE_PLATFORM_NAME = "Android" ]; then
# Install Flutter
curl https://storage.googleapis.com/flutter_infra/releases/stable/linux/flutter_linux_$FLUTTER_VERSION.tar.xz -o flutter_linux.tar.xz
tar xf flutter_linux.tar.xz
fi
if [ $DEVICEFARM_DEVICE_PLATFORM_NAME = "iOS" ]; then
# Install Flutter
curl https://storage.googleapis.com/flutter_infra/releases/stable/macos/flutter_macos_$FLUTTER_VERSION.zip -o flutter_macos.zip
unzip -qq flutter_macos.zip
fi
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 485M 0 1198k 0 0 3201k 0 0:02:35 --:--:-- 0:02:35 3196k
5 485M 5 28.4M 0 0 20.6M 0 0:00:23 0:00:01 0:00:22 20.6M
15 485M 15 75.2M 0 0 31.6M 0 0:00:15 0:00:02 0:00:13 31.6M
24 485M 24 119M 0 0 35.2M 0 0:00:13 0:00:03 0:00:10 35.2M
31 485M 31 151M 0 0 34.6M 0 0:00:13 0:00:04 0:00:09 34.6M
39 485M 39 193M 0 0 35.9M 0 0:00:13 0:00:05 0:00:08 38.3M
49 485M 49 239M 0 0 37.5M 0 0:00:12 0:00:06 0:00:06 42.1M
57 485M 57 280M 0 0 37.9M 0 0:00:12 0:00:07 0:00:05 40.8M
67 485M 67 325M 0 0 38.8M 0 0:00:12 0:00:08 0:00:04 41.2M
75 485M 75 365M 0 0 38.9M 0 0:00:12 0:00:09 0:00:03 42.7M
84 485M 84 412M 0 0 39.7M 0 0:00:12 0:00:10 0:00:02 43.8M
92 485M 92 450M 0 0 39.5M 0 0:00:12 0:00:11 0:00:01 42.1M
100 485M 100 485M 0 0 39.3M 0 0:00:12 0:00:12 --:--:-- 41.6M
[DeviceFarm] mv flutter $HOME/flutter
[DeviceFarm] export PATH=$PATH:$HOME/flutter/bin:$HOME/flutter/bin/cache/dart-sdk/bin
[DeviceFarm] export PATH="$PATH":"$HOME/.pub-cache/bin"
[DeviceFarm] flutter doctor -v
Warning! This package referenced a Flutter repository via the .packages file that is no longer available. The repository from which the 'flutter' tool is currently executing will be used instead.
running Flutter tool: /home/device-farm/flutter
previous reference : /C:/flutter-sdk
This can happen if you deleted or moved your copy of the Flutter repository, or if it was on a volume that is no longer mounted or has been mounted at a different location. Please check your system path to verify that you are running the expected version (run 'flutter --version' to see which flutter is on your path).
╔════════════════════════════════════════════════════════════════════════════╗
║ A new version of Flutter is available! ║
║ ║
║ To update to the latest version, run "flutter upgrade". ║
╚════════════════════════════════════════════════════════════════════════════╝
╔════════════════════════════════════════════════════════════════════════════╗
║ Welcome to Flutter! - https://flutter.dev ║
║ ║
║ The Flutter tool uses Google Analytics to anonymously report feature usage ║
║ statistics and basic crash reports. This data is used to help improve ║
║ Flutter tools over time. ║
║ ║
║ Flutter tool analytics are not sent on the very first run. To disable ║
║ reporting, type 'flutter config --no-analytics'. To display the current ║
║ setting, type 'flutter config'. If you opt out of analytics, an opt-out ║
║ event will be sent, and then no further information will be sent by the ║
║ Flutter tool. ║
║ ║
║ By downloading the Flutter SDK, you agree to the Google Terms of Service. ║
║ Note: The Google Privacy Policy describes how data is handled in this ║
║ service. ║
║ ║
║ Moreover, Flutter includes the Dart SDK, which may send usage metrics and ║
║ crash reports to Google. ║
║ ║
║ Read about data we send with crash reports: ║
║ https://github.com/flutter/flutter/wiki/Flutter-CLI-crash-reporting ║
║ ║
║ See Google's privacy policy: ║
║ https://www.google.com/intl/en/policies/privacy/ ║
╚════════════════════════════════════════════════════════════════════════════╝
[✓] Flutter (Channel stable, v1.12.13+hotfix.8, on Linux, locale en_US.UTF-8)
• Flutter version 1.12.13+hotfix.8 at /home/device-farm/flutter
• Framework revision 0b8abb4724 (10 months ago), 2020-02-11 11:44:36 -0800
• Engine revision e1e6ced81d
• Dart version 2.7.0
[!] Android toolchain - develop for Android devices (Android SDK version 21.1.1)
• Android SDK at /opt/dev/android-sdk-linux
• Android NDK location not configured (optional; useful for native profiling support)
✗ Flutter requires Android SDK 28 and the Android BuildTools 28.0.3
To update using sdkmanager, run:
"/opt/dev/android-sdk-linux/tools/bin/sdkmanager" "platforms;android-28" "build-tools;28.0.3"
or visit https://flutter.dev/setup/#android-setup for detailed instructions.
! Some Android licenses not accepted. To resolve this, run: flutter doctor --android-licenses
[!] Android Studio (not installed)
• Android Studio not found; download from https://developer.android.com/studio/index.html
(or visit https://flutter.dev/setup/#android-setup for detailed instructions).
[✓] Connected device (1 available)
• Pixel 3 • 8AGX0SK0H • android-arm64 • Android 10 (API 29)
! Doctor found issues in 2 categories.
[DEVICEFARM] ########### Entering phase pre_test ###########
[DEVICEFARM] ########### Entering phase test ###########
[DeviceFarm] TEST_PHASE_STATUS=0
[DeviceFarm] track_test_phase_status() {
local command_exit_code=$?
if [ "$command_exit_code" != "0" ]; then
echo "Most recent command failed with error code: $command_exit_code"
TEST_PHASE_STATUS=$command_exit_code
fi
}
set_test_phase_exit_code() {
return $TEST_PHASE_STATUS
}
[DeviceFarm] echo "Navigate to test package directory"
Navigate to test package directory
[DeviceFarm] cd $DEVICEFARM_TEST_PACKAGE_PATH
[DeviceFarm] echo "Start Flutter integration test"
Start Flutter integration test
[DeviceFarm] cd flutter_app
[DeviceFarm] rm -f .packages
[DeviceFarm] MAIN=test_driver/main.dart
[DeviceFarm] TESTS=test_driver/main_test.dart
[DeviceFarm] if [ $DEVICEFARM_DEVICE_PLATFORM_NAME = "Android" ]; then
./script/test_android.sh --run-tests "$TESTS"
track_test_phase_status
fi
if [ $DEVICEFARM_DEVICE_PLATFORM_NAME = "iOS" ]; then
./script/test_ios.sh --unpack $DEVICEFARM_APP_PATH
track_test_phase_status
./script/test_ios.sh --dummy-symbols build_to_os.txt
track_test_phase_status
./script/test_ios.sh --run-tests "$MAIN" "$TESTS"
track_test_phase_status
fi
Starting Flutter app com.example.sylph_aws_farm_test1
in debug mode...
Running "flutter pub get" in flutter_app...
The current Dart SDK version is 2.7.0.
Because sylph_aws_farm_test1 depends on cupertino_icons >=1.0.1 which requires SDK version >=2.12.0-0 <3.0.0, version solving failed.
pub get failed (1; Because sylph_aws_farm_test1 depends on cupertino_icons >=1.0.1 which requires SDK version >=2.12.0-0 <3.0.0, version solving failed.)
Most recent command failed with error code: 1
[DeviceFarm] for f in flutter_*.log; do
if [ -e "$f" ]; then
echo "Archiving $f to $DEVICEFARM_LOG_DIR"
mv $f $DEVICEFARM_LOG_DIR
fi
done
[DeviceFarm] ls -la $DEVICEFARM_LOG_DIR
total 44
drwxrwxrwx 2 rwx rwx 4096 Nov 23 07:14 .
drwxrwxrwx 12 root root 36864 Nov 23 07:15 ..
[DeviceFarm] echo done.
done.
[DeviceFarm] set_test_phase_exit_code
[DEVICEFARM] ########### Entering phase post_test ###########
[DEVICEFARM] ########### Finish executing testspec ###########
[DEVICEFARM] ########### Setting upload permissions ###########
chmod: cannot access ‘/tmp/customer_log_directory0mOCNu/*’: No such file or directory
[DEVICEFARM] Tearing down your device. Your tests report will come shortly.