Skip to content

Commit 9370656

Browse files
authored
chore: Create and use CMB Service Yamato job (#3445)
<!-- Replace this block with what this PR does and why. Describe what you'd like reviewers to know, how you applied the engineering principles, and any interesting tradeoffs made. Delete bullet points below that don't apply, and update the changelog section as appropriate. --> <!-- Add short version of the JIRA ticket to the PR title (e.g. "feat: new shiny feature [MTT-123]") --> ## Changelog - Added: A separate yamato job for running the CMB Service tests ## Testing and Documentation - Updated pr trigger to run all standalone windows tests and then cmb service ubuntu tests. <!-- Uncomment and mark items off with a * if this PR deprecates any API: ### Deprecated API - [ ] An `[Obsolete]` attribute was added along with a `(RemovedAfter yyyy-mm-dd)` entry. - [ ] An [api updater] was added. - [ ] Deprecation of the API is explained in the CHANGELOG. - [ ] The users can understand why this API was removed and what they should use instead. --> ## Backport <!-- If this is a backport: - Add the following to the PR title: "\[Backport\] ..." . - Link to the original PR. If this needs a backport - state this here If a backport is not needed please provide the reason why. If the "Backports" section is not present it will lead to a CI test failure. --> No backport required as this is related to cmb service which is 2.X only
1 parent 46424d1 commit 9370656

File tree

3 files changed

+76
-28
lines changed

3 files changed

+76
-28
lines changed

.yamato/_triggers.yml

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -50,13 +50,14 @@ pull_request_trigger:
5050
- .yamato/_run-all.yml#run_all_package_tests_trunk
5151
# Run package EditMode and Playmode package tests on minimum supported editor (6000.0 in case of NGOv2.X)
5252
- .yamato/_run-all.yml#run_all_package_tests_6000
53-
# Run project EditMode and PLaymode project tests on trunk
53+
# Run project EditMode and Playmode project tests on trunk
5454
- .yamato/_run-all.yml#run_all_project_tests_trunk
55-
# Run project EditMode and PLaymode project tests on minimum supported editor (6000.0 in case of NGOv2.X)
55+
# Run project EditMode and Playmode project tests on minimum supported editor (6000.0 in case of NGOv2.X)
5656
- .yamato/_run-all.yml#run_all_project_tests_6000
57-
# Run standalone test. We run it only on Ubuntu since it's the fastest machine, and it was noted that for example distribution on macOS is taking 40m since we switched to Apple Silicon
57+
# Run standalone test. Run windows standalone tests as this is our most common platform, and run ubuntu rust tests.
5858
# Coverage on other standalone machines is present in Nightly job so it's enough to not run all of them for PRs
59-
- .yamato/desktop-standalone-tests.yml#desktop_standalone_test_testproject_ubuntu_il2cpp_6000.0
59+
- .yamato/desktop-standalone-tests.yml#desktop_standalone_test_testproject_win_il2cpp_6000.0
60+
- .yamato/cmb-service-standalone-tests.yml#cmb_service_standalone_test_testproject_ubuntu_il2cpp_6000.0
6061
triggers:
6162
cancel_old_ci: true
6263
pull_requests:
@@ -133,4 +134,4 @@ develop_weekly_trunk:
133134
# Build player for webgl platform on trunk
134135
- .yamato/_run-all.yml#run_all_webgl_builds
135136
# Run code coverage test
136-
- .yamato/code-coverage.yml#code_coverage_ubuntu_trunk
137+
- .yamato/code-coverage.yml#code_coverage_ubuntu_trunk
Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
{% metadata_file .yamato/project.metafile %} # All configuration that is used to create different configurations (used in for loops) is taken from this file.
2+
---
3+
4+
# DESCRIPTION--------------------------------------------------------------------------
5+
# This job is responsible test validation against the CMB Service.
6+
# This job runs on the default test_platform which is Ubuntu.
7+
# This job runs the standalone desktop tests and ignores all tests that are not related to the CMB Service.
8+
# Those tests cover both PlayMode and EditMode tests from package test assemblies.
9+
# This is only needed in NGOv2.X since NGOv1.X does not use the CMB Service.
10+
11+
# CONFIGURATION STRUCTURE--------------------------------------------------------------
12+
# Jobs are generated using nested loops (separate build phase and run phase). Worth noting that run phase uses the build as dependency:
13+
# 1. For all desktop platform (Windows, macOS, Ubuntu)
14+
# 2. For all supported Unity Editor versions (for NGOv2.X this means 6000.0+ editors)
15+
# 3. For the default project.
16+
# 4. For all scripting backends (mono, il2cpp)
17+
18+
# TECHNICAL CONSIDERATIONS---------------------------------------------------------------
19+
# For desktop devices a split is into two phases is not required but we use it for consistency with setup of others standalone platforms:
20+
# 1. Build Phase: Creates standalone players for desktop platforms
21+
# 2. Run Phase: Executes runtime tests on actual desktop devices
22+
# The Run phase uses build job as dependency
23+
24+
# Note: The build phase for this test is imported from the desktop standalone tests as the CMB Service test is a subset of the desktop standalone tests.
25+
26+
#-----------------------------------------------------------------------------------
27+
28+
29+
# CMB SERVICE RUN PHASE CONFIGURATION------------------------------------------------------------------------------------
30+
{% for project in projects.default -%}
31+
{% for platform in test_platforms.default -%}
32+
{% for editor in validation_editors.all -%}
33+
{% for backend in scripting_backends -%}
34+
cmb_service_standalone_test_{{ project.name }}_{{ platform.name }}_{{ backend }}_{{ editor }}:
35+
name : CMB Service Test - NGO {{ project.name }} - [{{ platform.name }}, {{ editor }}, {{ backend }}]
36+
agent:
37+
type: {{ platform.type }}::GPU
38+
image: {{ platform.image }}
39+
flavor: {{ platform.flavor }}
40+
41+
variables:
42+
# The echo server is a "mock" server that is only used to test encoding/decoding of messages.
43+
# It is used by the DistributedAuthorityCodecTests. These are tests that are built and maintained by the CMB service team.
44+
ECHO_SERVER_PORT: "7788"
45+
# Set this to ensure the DistributedAuthorityCodecTests will fail if they cannot connect to the echo server.
46+
# The default is to ignore the codec tests if the echo server fails to connect
47+
ENSURE_CODEC_TESTS: "true"
48+
49+
# When USE_CMB_SERVICE is set to true, any C# tests configured to use the DA host will instead use the CMB service.
50+
USE_CMB_SERVICE: "true"
51+
# This is the port on which to run the full standalone CMB service.
52+
# The port needs to be different from the echo server port as two processes cannot bind to same port.
53+
CMB_SERVICE_PORT: "7799"
54+
55+
commands:
56+
# run_cmb_service.sh builds and starts a release version of the full CMB service (along with the limited echo server)
57+
- ./Tools/CI/run_cmb_service.sh -e $ECHO_SERVER_PORT -s $CMB_SERVICE_PORT
58+
59+
- unity-downloader-cli --fast --wait -u {{ editor }} -c Editor {% if backend == "il2cpp" %} -c il2cpp {% endif %} {% if platform.name == "mac" %} --arch arm64 {% endif %} # For macOS we use ARM64 models
60+
- UnifiedTestRunner --suite=playmode --player-load-path=build/players --artifacts-path=test-results --testproject={{ project.path }} --editor-location=.Editor --playergraphicsapi=Null --fail-on-assert --reruncount=1 --clean-library-on-rerun --timeout=1800
61+
artifacts:
62+
logs:
63+
paths:
64+
- "test-results/**/*"
65+
dependencies:
66+
- .yamato/desktop-standalone-tests.yml#desktop_standalone_build_{{ project.name }}_{{ platform.name }}_{{ backend }}_{{ editor }}
67+
{% endfor -%}
68+
{% endfor -%}
69+
{% endfor -%}
70+
{% endfor -%}

.yamato/desktop-standalone-tests.yml

Lines changed: 0 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -77,30 +77,7 @@ desktop_standalone_test_{{ project.name }}_{{ platform.name }}_{{ backend }}_{{
7777
model: {{ platform.model }} # This is set only in platforms where we want non-default model to use (more information in project.metafile). In this case it's specifically for macOS (if used for win/ubuntu then it will cause rust server to fail connecting)
7878
{% endif %}
7979

80-
# Set additional variables for running the echo server (This is needed ONLY for NGOv2.X because relates to Distributed Authority)
81-
{% if platform.name != "win" %} # Issues with win and mac are tracked in MTT-11606
82-
variables:
83-
# The echo server is a "mock" server that is only used to test encoding/decoding of messages.
84-
# It is used by the DistributedAuthorityCodecTests. These are tests that are built and maintained by the CMB service team.
85-
ECHO_SERVER_PORT: "7788"
86-
# Set this to ensure the DistributedAuthorityCodecTests will fail if they cannot connect to the echo server.
87-
# The default is to ignore the codec tests if the echo server fails to connect
88-
ENSURE_CODEC_TESTS: "true"
89-
90-
# When USE_CMB_SERVICE is set to true, any C# tests configured to use the DA host will instead use the CMB service.
91-
USE_CMB_SERVICE: "true"
92-
# This is the port on which to run the full standalone CMB service.
93-
# The port needs to be different from the echo server port as two processes cannot bind to same port.
94-
CMB_SERVICE_PORT: "7799"
95-
{% endif %}
96-
9780
commands:
98-
# If ubuntu, run rust echo server (This is needed ONLY for NGOv2.X because relates to Distributed Authority)
99-
{% if platform.name != "win" %} # Issues with win and mac are tracked in MTT-11606
100-
# run_cmb_service.sh builds and starts a release version of the full CMB service (along with the limited echo server)
101-
- ./Tools/CI/run_cmb_service.sh -e $ECHO_SERVER_PORT -s $CMB_SERVICE_PORT
102-
{% endif %}
103-
10481
- unity-downloader-cli --fast --wait -u {{ editor }} -c Editor {% if backend == "il2cpp" %} -c il2cpp {% endif %} {% if platform.name == "mac" %} --arch arm64 {% endif %} # For macOS we use ARM64 models
10582
- UnifiedTestRunner --suite=playmode --player-load-path=build/players --artifacts-path=test-results --testproject={{ project.path }} --editor-location=.Editor --playergraphicsapi=Null --fail-on-assert --reruncount=1 --clean-library-on-rerun --timeout=1800
10683
artifacts:

0 commit comments

Comments
 (0)