UI Testing Support with IntelliJ IDE Starter #537
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,6 +1,6 @@ | ||
| # GitHub Actions Workflow for launching UI tests on Linux and Mac in the following steps: | ||
| # - Prepare and launch IDE with your plugin and robot-server plugin, which is necessary to interact with the UI. | ||
| # - Wait for IDE to start | ||
| # - Run UI tests with a separate Gradle task. | ||
| # | ||
| # Please check https://github.com/JetBrains/intellij-ui-test-robot for information about UI tests with IntelliJ Platform. | ||
|
|
@@ -12,28 +12,28 @@ on: | |
| workflow_dispatch | ||
|
|
||
| jobs: | ||
| # UI Test Jobs (one per OS/test combination) | ||
| testUI: | ||
| name: Run UI Test on ${{ matrix.osName }} | ||
| runs-on: ${{ matrix.runner }} | ||
| permissions: | ||
| contents: write | ||
| checks: write | ||
| pull-requests: write | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| include: | ||
| # Linux test | ||
| - runner: ubuntu-latest | ||
| osName: Linux | ||
| # macOS test | ||
| - runner: macos-latest | ||
| osName: macOS | ||
|
Contributor
Author
There was a problem hiding this comment. I had firewall issues when running on Windows ( also with the previous Robot Framework) |
||
|
|
||
| steps: | ||
| - name: Fetch Sources | ||
| uses: actions/checkout@v5 | ||
|
|
||
| # Set up the Java environment for the next steps | ||
| - name: Setup Java | ||
|
|
@@ -44,22 +44,100 @@ jobs: | |
|
|
||
| # Setup Gradle | ||
| - name: Setup Gradle | ||
| uses: gradle/actions/setup-gradle@v4.2.2 | ||
| with: | ||
| cache-read-only: true | ||
|
|
||
| # Set up Xvfb for Linux headless mode | ||
| - name: Setup Xvfb for Linux | ||
| if: runner.os == 'Linux' | ||
| run: | | ||
| sudo apt-get update -y | ||
| sudo apt-get install -y xvfb x11-utils libxkbcommon-x11-0 libxcb-icccm4 libxcb-image0 libxcb-keysyms1 libxcb-randr0 libxcb-render-util0 libxcb-xinerama0 libxcb-xfixes0 | ||
| export DISPLAY=:99.0 | ||
| Xvfb -ac :99 -screen 0 1920x1080x16 & | ||
| sleep 3 | ||
| echo "DISPLAY=:99.0" >> $GITHUB_ENV | ||
|
|
||
| # Run UI test with IDE Starter framework | ||
| - name: Run UI Test | ||
| env: | ||
| CODECOV_API_TOKEN: ${{ secrets.CODECOV_API_TOKEN }} | ||
| LICENSE_KEY: ${{ secrets.LICENSE_KEY }} | ||
| UI_BRANCH_NAME: "${{ matrix.osName }}-${{ github.ref_name }}" | ||
| DISPLAY: ${{ runner.os == 'Linux' && ':99.0' || '' }} | ||
| run: | | ||
| echo "UI on ${{ matrix.osName }} run #: ${{ github.run_number }} and id: ${{ github.run_id }}" | ||
| ./gradlew uiTest | ||
| timeout-minutes: 30 | ||
|
|
||
| # List build directory for debugging | ||
| - name: List build directory structure | ||
| if: always() | ||
| shell: bash | ||
| run: | | ||
| echo "=== Build directory structure ===" | ||
| if [ -d "build" ]; then | ||
| find build -type d -name "log" -o -name "screenshots" -o -name "ui-hierarchy" 2>/dev/null | head -20 || true | ||
| echo "=== Looking for IDE test output ===" | ||
| find build -type d -path "*/ide-tests/*" 2>/dev/null | head -20 || true | ||
| else | ||
| echo "Build directory not found" | ||
| fi | ||
|
|
||
| - name: Collect Test Results | ||
|
Contributor
Author
There was a problem hiding this comment. This can be removed if we update SetUp.kt to use full CI integration with the DI framework. |
||
| if: always() | ||
| uses: actions/[email protected] | ||
| with: | ||
| name: tests-result-${{ matrix.osName }} | ||
| path: | | ||
| build/reports/ | ||
| build/test-results/ | ||
| build/out/ | ||
| build/idea-sandbox/ | ||
| build/**/*.log | ||
| build/**/screenshots/ | ||
| build/**/ui-hierarchy/ | ||
| if-no-files-found: warn | ||
| continue-on-error: true | ||
|
|
||
| - name: Collect Test Report XML | ||
| if: always() | ||
| uses: actions/[email protected] | ||
| with: | ||
| name: test-report-${{ matrix.osName }} | ||
| path: ${{ github.workspace }}/build/test-results/uiTest/*.xml | ||
|
|
||
| # Test report job using dorny/test-reporter | ||
| test-report: | ||
| name: UI Test Report | ||
| needs: testUI | ||
| runs-on: ubuntu-latest | ||
| if: always() | ||
| permissions: | ||
| contents: write | ||
| checks: write | ||
| pull-requests: write | ||
| actions: read | ||
|
|
||
| steps: | ||
| - name: Fetch Sources | ||
| uses: actions/checkout@v5 | ||
| with: | ||
| token: '${{ secrets.GITHUB_TOKEN }}' | ||
|
|
||
| - name: Download all test reports | ||
| uses: actions/[email protected] | ||
| with: | ||
| pattern: test-report-* | ||
| path: test-reports | ||
| merge-multiple: true | ||
|
|
||
| - name: Generate Test Report | ||
| uses: dorny/[email protected] | ||
| if: success() || failure() | ||
| with: | ||
| name: UI Test Report Dashboard | ||
| reporter: java-junit | ||
| fail-on-error: 'true' | ||
| path: 'test-reports/*.xml' | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| <component name="ProjectRunConfigurationManager"> | ||
| <configuration default="false" name="Run UI Tests" type="GradleRunConfiguration" factoryName="Gradle"> | ||
| <ExternalSystemSettings> | ||
| <option name="executionName" /> | ||
| <option name="externalProjectPath" value="$PROJECT_DIR$" /> | ||
| <option name="externalSystemIdString" value="GRADLE" /> | ||
| <option name="scriptParameters" value="" /> | ||
| <option name="taskDescriptions"> | ||
| <list /> | ||
| </option> | ||
| <option name="taskNames"> | ||
| <list> | ||
| <option value="uiTest" /> | ||
| </list> | ||
| </option> | ||
| <option name="vmOptions" /> | ||
| </ExternalSystemSettings> | ||
| <ExternalSystemDebugServerProcess>true</ExternalSystemDebugServerProcess> | ||
| <ExternalSystemReattachDebugProcess>true</ExternalSystemReattachDebugProcess> | ||
| <DebugAllEnabled>false</DebugAllEnabled> | ||
| <RunAsTest>false</RunAsTest> | ||
| <method v="2" /> | ||
| </configuration> | ||
| </component> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is needed only because we run the check - the test task already has the tag exclusion, so when running the test task in the IDE or Gradle, it runs only the unit-tests