Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
3dac016
Add custom providers documentation for the plugin system
frostebite Mar 5, 2026
d706850
Comprehensive orchestrator documentation overhaul
frostebite Mar 5, 2026
172ae89
Remove old standalone GitLab pages from versioned docs
frostebite Mar 5, 2026
c95128d
Restructure Orchestrator docs: promote Providers to top-level, improv…
frostebite Mar 5, 2026
07229a6
Merge Configuration Override into Secrets page, rename to Pull Secrets
frostebite Mar 5, 2026
6f9667f
Fix broken link: hooks directory has no index page
frostebite Mar 5, 2026
843fe49
Add comprehensive GitHub Actions examples page
frostebite Mar 5, 2026
e9abe77
Fix ASCII diagram alignment in Game-CI vs Orchestrator
frostebite Mar 5, 2026
d128415
Add ASCII diagrams to custom providers, GitHub integration, and retai…
frostebite Mar 5, 2026
759a16f
Add ASCII diagrams to container hooks, garbage collection, AWS, and p…
frostebite Mar 5, 2026
db90cbc
Replace provider decision tree with simple 4-across comparison
frostebite Mar 5, 2026
07da84a
Fix sidebar ordering: Secrets before Advanced Topics
frostebite Mar 5, 2026
557659d
Rename Premade Container Hooks to Built-In Hooks
frostebite Mar 5, 2026
9df05bc
Rename Built-In Hooks file, move Custom Job out of Hooks, fix alignment
frostebite Mar 5, 2026
b70b662
Fix alignment of all ASCII diagrams across orchestrator docs
frostebite Mar 5, 2026
c7ac032
Add Load Balancing documentation page
frostebite Mar 5, 2026
cf9039b
Add Storage, Architecture pages and Build Caching section
frostebite Mar 5, 2026
a997186
Fix introduction diagram to list all supported CI platforms
frostebite Mar 5, 2026
7b370f1
Fix ASCII diagram alignment in load balancing page
frostebite Mar 5, 2026
93c86da
docs: add Orchestrator Jobs page and Custom LFS Agents page
frostebite Mar 5, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
141 changes: 54 additions & 87 deletions docs/03-github-orchestrator/01-introduction.mdx
Original file line number Diff line number Diff line change
@@ -1,104 +1,71 @@
# Introduction

## Concept - What Does Orchestrator Do
## What Does Orchestrator Do?

**Orchestrator enables you to run, build and test (Unity) projects in the cloud. You can start jobs
from the command line, the "Workbench" GUI in the Unity Editor or from GitHub Actions.**
**Orchestrator runs Unity builds on cloud infrastructure.** Start jobs from GitHub Actions, the
command line, or any CI system. Orchestrator provisions a cloud environment, sends your project to
be built, and streams results back.

**Orchestrator will automatically provision an environment at a Cloud Provider such as GCP and AWS.
It will then send the project to be built and/or tested depending on your workflow configuration.**

**Orchestrator is especially useful for game development because it supports large projects.
Orchestrator provides first class support for the Unity game engine.**

Orchestrator uses git to track and syncronize your projects and uses native cloud services such as
AWS Fargate and Kubernetes to run your jobs. Other version control systems are not actively
supported.

## Why Orchestrator?

1. **Orchestrator is flexible and elastic**
1. _You can balance your use of high-performance and cost saving modes._ Configurable cost/speed
effeciency
2. _Works great for projects of almost any size, from AAA projects and assets to micro projects_
3. _Extended configuration options._ More control over disk size, memory and CPU
4. _Easily scale all job resources as your project grows_ e.g storage, CPU and memory
2. **Scale fully on demand from zero (not in use) to many concurrent jobs.** Benefits from
"pay-for-what-you-use" cloud billing models. We have made an effort to make sure that it costs
you nothing (aside from artifact and cache storage) while there are no builds running (no
guarantees)
3. **Easy setup and configuration**
4. **Run custom jobs** and extend the system for any workload

## Why not orchestrator?

1. Your project is small in size. Below 5GB Orchestrator should not be needed.
2. You already have dedicated servers running you can use.

Although the speed of a CI pipelines is an important metric to consider, there are real challenges
for game development pipelines.

This solution prefers convenience, ease of use, scalability, throughput and flexibility.

Faster solutions exist, but would all involve self-hosted hardware with an immediate local cache of
the large project files and working directory and a dedicated server.

# Orchestrator Release Status

Orchestrator is in "active development" ⚠️🔨

Orchestrator overall release status: `preview` This means some APIs may change, features are still
being added but the minimum feature set works and is stable.

Release Stages: `experimental` ➡️ `preview` ➡️ `full release`

You must use a provider with Orchestrator, each provider's release status is described below. This
indicates the stability and support for orchestrator features and workflows.

### Supported Orchestrator Platforms

```md
| Cloud Provider Platform | Release Status |
| ----------------------- | ------------------ |
| Kubernetes | ✔️ preview release |
| AWS | ✔️ full release |
| GCP | ⚠ Considered |
| Azure | ⚠ Considered |
```
Your Machine / CI Cloud Provider
┌──────────────┐ git push ┌─────────────────┐
│ GitHub │ ─────────────────►│ AWS Fargate │
│ Actions, │ │ Kubernetes │
│ GitLab CI, │ ◄─────────────────│ Local Docker │
│ CLI, etc. │ build artifacts │ │
└──────────────┘ └─────────────────┘
│ │
│ Orchestrator handles: │
│ * Provisioning │
│ * Git sync + LFS │
│ * Caching (S3 / rclone) │
│ * Automatic cleanup │
└───────────────────────────────────┘
```

_Note for Kubernetes support:_ _Usually the cluster needs to be up and running at all times, as
starting up a cluster is slow._ _Use Google Cloud's Kubernetes Autopilot you can scale down to the
free tier automatically while not in use._ _Kubernetes support currently requires cloud storage,
only S3 support is built-in._
Orchestrator supports large projects with first-class Unity support and native cloud services like
AWS Fargate and Kubernetes.

```md
| Git Platform | Release Status |
| --------------------- | ------------------ |
| GitHub | ✔️ full release |
| GitLab | ✔️ preview release |
| Command Line | ✔️ preview release |
| Any Git repository | ✔️ preview release |
| Any Git automation/Ci | ✔️ preview release |
```
## ✅ Why Orchestrator?

## External Links
1. **Flexible and elastic** — balance speed and cost, configure CPU, memory, and disk per build
2. **Scale from zero** — no idle servers, pay only while builds run
3. **Easy setup** — minimal configuration to [get started](getting-started)
4. **Extensible** — run [custom hooks](advanced-topics/hooks/container-hooks), or bring your own
[provider plugin](providers/custom-providers)

### Orchestrator Releases
## ❌ When You Don't Need It

[Game CI Releases - GitHub](https://github.com/game-ci/unity-builder/releases) _Packaged and
released with game-ci's unity-builder module._
- Your project is under 5 GB — standard GitHub runners should work fine
- You have dedicated build servers already running

### Open Incoming Pull Requests
## 📦 Supported Providers

[Orchestrator PRs - GitHub](https://github.com/game-ci/unity-builder/pulls?q=is%3Apr+orchestrator)
| Cloud Provider | Description |
| -------------------------------------- | -------------------------------------------------------- |
| [AWS Fargate](providers/aws) | Fully managed containers on AWS. No servers to maintain. |
| [Kubernetes](providers/kubernetes) | Run on any Kubernetes cluster. |
| [Local Docker](providers/local-docker) | Docker containers on the local machine. |
| [Local](providers/local) | Direct execution on the host machine. |

### 💬Suggestions and 🐛Bugs (GitHub Issues):
See [Providers](providers/overview) for the full list including [custom](providers/custom-providers)
and [community](providers/community-providers) providers.

[Game CI Issues - GitHub](https://github.com/game-ci/unity-builder/labels/orchestrator)
## 🖥️ Supported Platforms

### Community
| Platform | Description |
| ---------------------------------------------- | ------------------------------------- |
| [GitHub Actions](providers/github-integration) | First-class support with Checks. |
| [GitLab CI](providers/gitlab-integration) | Via the Command Line mode. |
| [Command Line](examples/command-line) | Run from any terminal or script. |
| Any CI system | Anything that can run shell commands. |

**Share your feedback with us!**
## 🔗 External Links

- [**Discord Channel**](https://discord.com/channels/710946343828455455/789631903157583923)
- [**Feedback Form**](https://forms.gle/3Wg1gGf9FnZ72RiJ9)
- [Releases](https://github.com/game-ci/unity-builder/releases) — packaged with
game-ci/unity-builder
- [Pull Requests](https://github.com/game-ci/unity-builder/pulls?q=is%3Apr+orchestrator) — open
orchestrator PRs
- [Issues](https://github.com/game-ci/unity-builder/labels/orchestrator) — bugs and feature requests
- [Discord](https://discord.com/channels/710946343828455455/789631903157583923) — community chat
- [Feedback Form](https://forms.gle/3Wg1gGf9FnZ72RiJ9) — share your experience
64 changes: 34 additions & 30 deletions docs/03-github-orchestrator/02-game-ci-vs-orchestrator.mdx
Original file line number Diff line number Diff line change
@@ -1,42 +1,46 @@
# Game-CI vs Orchestrator
---
sidebar_label: Game-CI vs Orchestrator
---

# Standard Game-CI (Use Cases)
# Standard Game-CI vs Orchestrator Mode

The Game CI core is a maintained set of docker images that can be used to run workloads in many
scenarios.
## 🎮 Standard Game-CI

Game CI also provides specific GitHub actions for running workflows on GitHub. And a similar
workflow for running Game CI on GitLab and Circle CI. _All of these options use the build server
resources provided by those systems, this can be a constraint or very convenient depending on the
size of your project and the workloads you need to run._
Game CI provides Docker images and GitHub Actions for running Unity workflows on the build server
resources provided by your CI platform (GitHub, GitLab, Circle CI).

# Orchestrator (Use Cases)
**Best for:** Small to medium projects that fit within GitHub's resource limits.

## Sending Builds to the cloud
## ☁️ Orchestrator Mode

You may want to take advantage of cloud resources for lots of reasons (scale, speed, cost,
flexibility) or may want to start remote builds from the command line without slowing down your
development machine. Orchestrator can help you do this.
Orchestrator sends builds to cloud infrastructure (AWS Fargate, Kubernetes) instead of running on
the CI runner itself. This is useful when:

This may be a preference, more effecient or you may want to use systems that struggle to handle
large game development projects (GitHub being a good example).
- Your project **exceeds disk space limits** on GitHub-hosted runners
- You need **more CPU or memory** than the CI platform provides
- You want to **scale to many concurrent builds** without managing servers

### Large GitHub Projects
```
Standard Game-CI Orchestrator Mode

GitHub Actions by default run on
[build machines provided by GitHub](https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners).
For Unity projects the available disk size is quite small. You may experience an error related to
running out of disk space. You may also want to run the build on a server with more memory or
processing resources.
┌────────────────┐ ┌─────────────────┐ ┌──────────────┐
│ GitHub │ │ GitHub Action │ │ │
│ Runner │ │ CLI / Any CI │───►│ Cloud │
│ │ │ │ │ Container │
│ (builds │ │ (dispatches │◄───│ (builds │
│ locally) │ │ only) │ │ remotely) │
└────────────────┘ └─────────────────┘ └──────────────┘
~14 GB disk Configurable CPU, memory, disk
Fixed resources Scales to zero when idle
```

### GitHub Self-Hosted Runners vs Game CI Orchestrator
## Self-Hosted Runners vs Orchestrator

_GitHub users can consider:
[GitHub self-hosted runners](https://docs.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners)
and Orchestrator. Both can enable you to build larger projects._
Both options let you build larger projects. Here's when to pick which:

_Orchestrator is better if you don't have a server setup or don't want to manage or maintain your
own build server._

_Self-hosted runners are best used when you already have a server available, running 24/7 that you
can setup as a runner. And you're happy to maintain and keep that server available and running._
| | Self-Hosted Runners | Orchestrator |
| --------------- | ---------------------------------- | ------------------------------------- |
| **Setup** | Requires a dedicated server | Cloud account + credentials |
| **Maintenance** | You manage the server 24/7 | Fully managed, no servers to maintain |
| **Cost model** | Fixed (server always running) | Pay per build (scales to zero) |
| **Best for** | Teams with existing infrastructure | Teams without dedicated servers |
9 changes: 5 additions & 4 deletions docs/03-github-orchestrator/02-getting-started.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,19 @@ runners. This is useful for large projects that exceed GitHub's disk or resource

- A Unity project in a GitHub repository
- A cloud provider account (AWS or a Kubernetes cluster)
- Provider credentials configured as GitHub secrets
- Provider credentials configured as GitHub [secrets](secrets)

## Quick Start

1. **Choose a provider**: `aws` (AWS Fargate) or `k8s` (Kubernetes)
2. **Configure credentials** for your chosen provider
3. **Add the orchestrator step** to your workflow

See the provider-specific examples for complete setup:
See the provider-specific setup guides:

- [AWS Example](examples/github-examples/aws)
- [Kubernetes Example](examples/github-examples/kubernetes)
- [AWS Fargate](providers/aws)
- [Kubernetes](providers/kubernetes)
- [Local Docker](providers/local-docker)
- [Command Line](examples/command-line)

## Minimal Example
Expand Down
68 changes: 43 additions & 25 deletions docs/03-github-orchestrator/03-examples/01-command-line.mdx
Original file line number Diff line number Diff line change
@@ -1,49 +1,67 @@
# Command Line

_Preview Support Only_

You can install Game CI locally and start orchestrator jobs from the command line or by integrating
your own tools. All parameters in [API Reference](../api-reference) can be specified as command line
input fields.

# Install
your own tools. All parameters in the [API Reference](../api-reference) can be specified as CLI
flags.

Currently (development)
## Install

```bash
git clone https://github.com/game-ci/unity-builder.git
cd unity-builder
yarn install
yarn run cli -m {mode parameter} --projectPath {Your project path} {... other command line parameters}
```

# Planned (does not work currently)
## Usage

```bash
yarn run cli -m <mode> --projectPath <path> [options]
```

### Examples

Run a standard build:

```bash
yarn run cli -m cli-build \
--projectPath /path/to/your/unity/project \
--providerStrategy aws \
--targetPlatform StandaloneLinux64 \
--gitPrivateToken $GIT_TOKEN
```

List active resources:

```bash
yarn run cli -m list-resources --providerStrategy aws
```

We plan to offer support for Game CI via Deno. This will enable fast, typescript native runtime and
you will be able to access this via the following:
Watch a running build:

```bash
dpx game-ci build
yarn run cli -m watch --providerStrategy aws
```

# Help
Clean up old resources:

_You can run `yarn run cli -h` or `yarn run cli --help` to list all modes and paramters with
descriptions_
```bash
yarn run cli -m garbage-collect --providerStrategy aws
```

# Main Command Parameters
## Help

- Default: `cli` (runs a standard build workflow)
- See API Reference "Modes"
Run `yarn run cli --help` to list all modes and parameters with descriptions.

# Keeping command line parameters short
## Keeping Commands Short

You can avoid specifying long command line input for credentials by using environment variables or
[the input override feature](../advanced-topics/configuration-override#example) to shorten commands
signficantly.
You can avoid long CLI flags for credentials by using environment variables or the
[Pull Secrets](../secrets#-pulling-secrets-from-external-sources) feature.

This enables you to provide a command to pull input, e.g you can pull from a file or from a secret
manager.
This lets you pull input from a secret manager:

```bash
yarn run cli --populateOverride true --pullInputList UNITY_EMAIL,UNITY_SERIAL,UNITY_PASSWORD --inputPullCommand="gcloud secrets versions access 1 --secret=\"{0}\""
yarn run cli \
--populateOverride true \
--pullInputList UNITY_EMAIL,UNITY_SERIAL,UNITY_PASSWORD \
--inputPullCommand='gcloud secrets versions access 1 --secret="{0}"'
```
Loading
Loading