-
-
Notifications
You must be signed in to change notification settings - Fork 225
iOS & Android integration tests #4559
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 54 commits
Commits
Show all changes
58 commits
Select commit
Hold shift + click to select a range
46bbbfe
WIP: iOS integration test
jpnurmi 43b57c7
Split common.ps1 vs. setup.ps1
jpnurmi d79436c
Fix empty --device argument
jpnurmi a916900
Temp speed up
jpnurmi 67fd410
Try BeforeAll
jpnurmi 83c3084
Restore, retry, artifacts
jpnurmi 6cc7696
Add test/Sentry.Maui.Device.IntegrationTestApp
jpnurmi abdc057
Update solution filter
jpnurmi 7ac73d0
Speed up
jpnurmi adf7b68
Partial revert
jpnurmi e190195
Managed vs. Native vs. NullRefenceException
jpnurmi aff8901
Fixup
jpnurmi 29bef65
Move to integration-test/ios.Tests.ps1
jpnurmi 2c516c2
Fixup
jpnurmi 30b08fd
Use shared Install-XHarness
jpnurmi 6389645
Fix Install-XHarness
jpnurmi 44a190e
Add RunTestApp helper func
jpnurmi 26e6d05
Fix Install-XHarness - attempt 2
jpnurmi 5f6eb95
Prefer booted sims
jpnurmi 28a57ae
Convert tabs to spaces
jpnurmi ba70ab2
Fix PreferredStates
jpnurmi 64d2827
Fix Install-XHarness
jpnurmi 90cda33
Rename to mobile.Tests.ps1 and prepare iOS context
jpnurmi c58bd4f
Leave scripts/device-test-utils.ps1 in the sparse checkout for integr…
jpnurmi 5e01b48
Add missing timeout
jpnurmi 8121c4c
Fix Android tfm
jpnurmi d508359
Prepare predictable Activity name on Android
jpnurmi 76eeddf
Fix path
jpnurmi 4931fcb
Add missing continue-on-error: true
jpnurmi f438a74
Apply suggestions from code review
jpnurmi 459508a
Move the app to integration-test/mobile-app
jpnurmi 24646df
restore ios 18.5
jpnurmi 9b924e3
MSBUILD : error MSB1009: Project file does not exist.
jpnurmi ad8f805
.gitignore :x
jpnurmi 6b88514
WIP: add integration tests for Android
jpnurmi 8716532
Run Android integration tests in CI
jpnurmi 2b3bb23
try catch for adb devices
jpnurmi 73274fb
Get-AndroidEmulatorId
jpnurmi fea7c1e
Split mobile.Tests.ps1 -> ios.Tests.ps1 vs. android.Tests.ps1
jpnurmi 3a4e786
Try checkout ../github-workflows
jpnurmi 1131252
modules/github-workflows
jpnurmi bab2235
setup env
jpnurmi 88b40b1
checkout submodules
jpnurmi 7c4f0b9
Try selecting Java 17 for integration tests
jpnurmi 8144e87
Clean up & use the new YAML anchors to reduce repeat
jpnurmi 2f2b515
Simplify iOS integration test skip condition
jpnurmi 185dbf2
Simplify and stabilize
jpnurmi 8f3b094
Move test assertions outside output groups
jpnurmi 97062d5
Replace unamge with cross-platform OS arch checks
jpnurmi e1cc0c5
Replace direct `adb` calls with `xharness android adb`
jpnurmi 7d72059
Pre-install & xharness adb devices
jpnurmi 5ae455d
Don't try to run adb devices in the CI
jpnurmi 7089ea5
Update android.Tests.ps1
jamescrosswell 18c7fc8
Get-AndroidEmulatorId: fix env:ANDROID_SERIAL check
jpnurmi 4e2cf5d
Apply suggestion from @Flash0ver
jpnurmi c26d952
Merge remote-tracking branch 'upstream/main' into test/ios-integration
jpnurmi a8c25a7
Try-finally for adb reverse
jpnurmi fed73a4
Use --remove-all for port-forwarding cleanup
jpnurmi File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,161 @@ | ||
# This file contains test cases for https://pester.dev/ | ||
Set-StrictMode -Version Latest | ||
$ErrorActionPreference = 'Stop' | ||
. $PSScriptRoot/pester.ps1 | ||
. $PSScriptRoot/../scripts/device-test-utils.ps1 | ||
|
||
BeforeDiscovery { | ||
# Skip Android integration tests unless an emulator has been already started | ||
# by Android Device Tests, or manually when testing locally. This avoids | ||
# slowing down non-Device Test CI builds further. | ||
Install-XHarness | ||
$script:emulator = Get-AndroidEmulatorId | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. TIL about |
||
} | ||
|
||
Describe 'MAUI app' -ForEach @( | ||
@{ tfm = "net9.0-android35.0" } | ||
) -Skip:(-not $script:emulator) { | ||
BeforeAll { | ||
Remove-Item -Path "$PSScriptRoot/mobile-app" -Recurse -Force -ErrorAction SilentlyContinue | ||
Copy-Item -Path "$PSScriptRoot/net9-maui" -Destination "$PSScriptRoot/mobile-app" -Recurse -Force | ||
Push-Location $PSScriptRoot/mobile-app | ||
|
||
function InstallAndroidApp | ||
{ | ||
param([string] $Dsn) | ||
$dsn = $Dsn.Replace('http://', 'http://key@') + '/0' | ||
|
||
# replace {{SENTRY_DSN}} in MauiProgram.cs | ||
(Get-Content MauiProgram.cs) ` | ||
-replace '\{\{SENTRY_DSN\}\}', $dsn ` | ||
| Set-Content MauiProgram.cs | ||
|
||
$arch = [System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture.ToString().ToLower() | ||
$rid = "android-$arch" | ||
|
||
Write-Host "::group::Build Sentry.Maui.Device.IntegrationTestApp.csproj" | ||
dotnet build Sentry.Maui.Device.IntegrationTestApp.csproj ` | ||
--configuration Release ` | ||
--framework $tfm ` | ||
--runtime $rid | ||
| ForEach-Object { Write-Host $_ } | ||
Write-Host '::endgroup::' | ||
$LASTEXITCODE | Should -Be 0 | ||
|
||
Write-Host "::group::Install bin/Release/$tfm/$rid/io.sentry.dotnet.maui.device.integrationtestapp-Signed.apk" | ||
xharness android install -v ` | ||
--app bin/Release/$tfm/$rid/io.sentry.dotnet.maui.device.integrationtestapp-Signed.apk ` | ||
--package-name io.sentry.dotnet.maui.device.integrationtestapp ` | ||
--output-directory=test_output | ||
| ForEach-Object { Write-Host $_ } | ||
Write-Host '::endgroup::' | ||
$LASTEXITCODE | Should -Be 0 | ||
} | ||
|
||
function RunAndroidApp | ||
{ | ||
param( | ||
[string] $Dsn, | ||
[string] $TestArg = 'None' | ||
) | ||
|
||
# Setup port forwarding for accessing sentry-server at 127.0.0.1:8000 from the emulator | ||
$port = $Dsn.Split(':')[2].Split('/')[0] | ||
xharness android adb -v -- reverse tcp:$port tcp:$port | ||
|
||
|
||
Write-Host "::group::Run Android app (TestArg=$TestArg)" | ||
xharness android adb -v ` | ||
-- shell am start -S -n io.sentry.dotnet.maui.device.integrationtestapp/.MainActivity ` | ||
-e SENTRY_TEST_ARG $TestArg | ||
| ForEach-Object { Write-Host $_ } | ||
Write-Host '::endgroup::' | ||
$LASTEXITCODE | Should -Be 0 | ||
|
||
do | ||
{ | ||
Write-Host "Waiting for app..." | ||
Start-Sleep -Seconds 1 | ||
|
||
$procid = (& xharness android adb -- shell pidof "io.sentry.dotnet.maui.device.integrationtestapp") -replace '\s', '' | ||
$activity = (& xharness android adb -- shell dumpsys activity activities) -match "io\.sentry\.dotnet\.maui\.device\.integrationtestapp" | ||
|
||
} while ($procid -and $activity) | ||
|
||
xharness android adb -v -- reverse --remove tcp:$port | ||
jpnurmi marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
} | ||
|
||
function UninstallAndroidApp | ||
{ | ||
Write-Host "::group::Uninstall io.sentry.dotnet.maui.device.integrationtestapp" | ||
xharness android uninstall -v ` | ||
--package-name io.sentry.dotnet.maui.device.integrationtestapp | ||
| ForEach-Object { Write-Host $_ } | ||
$LASTEXITCODE | Should -Be 0 | ||
Write-Host '::endgroup::' | ||
} | ||
|
||
# Helper to dump server stderr if the test server reported errors | ||
function Dump-ServerErrors { | ||
param( | ||
[Parameter(Mandatory)] | ||
$Result | ||
) | ||
if ($Result.HasErrors()) { | ||
Write-Host '::group::sentry-server stderr' | ||
$Result.ServerStdErr | ForEach-Object { Write-Host $_ } | ||
Write-Host '::endgroup::' | ||
} | ||
} | ||
} | ||
|
||
AfterAll { | ||
Pop-Location | ||
} | ||
|
||
AfterEach { | ||
UninstallAndroidApp | ||
} | ||
|
||
It 'Managed crash' { | ||
$result = Invoke-SentryServer { | ||
param([string]$url) | ||
InstallAndroidApp -Dsn $url | ||
RunAndroidApp -Dsn $url -TestArg "Managed" | ||
RunAndroidApp -Dsn $url | ||
} | ||
|
||
Dump-ServerErrors -Result $result | ||
$result.HasErrors() | Should -BeFalse | ||
$result.Envelopes() | Should -AnyElementMatch "`"type`":`"System.ApplicationException`"" | ||
$result.Envelopes() | Should -Not -AnyElementMatch "`"type`":`"SIGABRT`"" | ||
} | ||
|
||
It 'Java crash' { | ||
$result = Invoke-SentryServer { | ||
param([string]$url) | ||
InstallAndroidApp -Dsn $url | ||
RunAndroidApp -Dsn $url -TestArg "Java" | ||
RunAndroidApp -Dsn $url | ||
} | ||
|
||
Dump-ServerErrors -Result $result | ||
$result.HasErrors() | Should -BeFalse | ||
$result.Envelopes() | Should -AnyElementMatch "`"type`":`"RuntimeException`"" | ||
$result.Envelopes() | Should -Not -AnyElementMatch "`"type`":`"System.\w+Exception`"" | ||
} | ||
|
||
It 'Null reference exception' { | ||
$result = Invoke-SentryServer { | ||
param([string]$url) | ||
InstallAndroidApp -Dsn $url | ||
RunAndroidApp -Dsn $url -TestArg "NullReferenceException" | ||
RunAndroidApp -Dsn $url | ||
} | ||
|
||
Dump-ServerErrors -Result $result | ||
$result.HasErrors() | Should -BeFalse | ||
$result.Envelopes() | Should -AnyElementMatch "`"type`":`"System.NullReferenceException`"" | ||
# TODO: fix redundant RuntimeException (#3954) | ||
{ $result.Envelopes() | Should -Not -AnyElementMatch "`"type`":`"SIGSEGV`"" } | Should -Throw | ||
} | ||
jpnurmi marked this conversation as resolved.
Show resolved
Hide resolved
|
||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.