Skip to content

Commit d23d275

Browse files
authored
docs: add fan-out fan-in section to workflows page (#573)
## 📝 Description - Add Fan-Out, Fan-In section back from the old docs to the current docs - Add a Job Matrix video to the job pages See #572 for more information Note: waiting for YouTube video to be published ## ✅ Checklist - [X] I have tested this change - [ ] This change requires documentation update
1 parent 0852231 commit d23d275

File tree

18 files changed

+398
-17
lines changed

18 files changed

+398
-17
lines changed
137 KB
Loading
211 KB
Loading
241 KB
Loading
243 KB
Loading

docs/docs/using-semaphore/jobs.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1089,6 +1089,8 @@ It's not possible to use job parallelism at the same time as [job matrices](#mat
10891089

10901090
## Job matrix {#matrix}
10911091

1092+
<VideoTutorial title="How to use a Job Matrix" src="https://www.youtube.com/embed/jRpj2Pu5eak" />
1093+
10921094
A job matrix is a more advanced form of [job parallelism](#job-parallelism) where you can define multiple variables with different values and run all the possible permutations.
10931095

10941096
For example, let's say we want to test our application using three Node.js versions using npm and yarn

docs/docs/using-semaphore/workflows.md

Lines changed: 133 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -27,30 +27,159 @@ Before you can run a workflow in Semaphore you need:
2727
- A Semaphore project linked to the repository
2828
- One or more pipelines
2929

30-
The [project page](./projects#view-projects) shows all the recent workflows for the project.
30+
The [project page](./projects#view-projects) shows all the recent workflows for the project.
3131

3232
![Types of workflows](./img/workflow-types.jpg)
3333

3434
## Visual workflow editor {#workflow-editor}
3535

3636
<VideoTutorial title="How to use the workflow builder" src="https://www.youtube.com/embed/dg2jDQmYJ_4" />
3737

38-
You can define most aspects of your workflows using the visual editor.
38+
You can define most aspects of your workflows using the visual editor.
3939

4040
To access the editor, open one of your projects on and press **Edit Workflow**. All your changes are stored as YAML pipeline definitions on your Git repository. You can make changes using the visual editor, or edit the YAML directly by clicking on the pipeline YAML file.
4141

4242
![Workflow editor button](./img/workflow-editor.jpg)
4343

4444
See the [jobs page](./jobs) to learn how to define jobs and blocks.
4545

46+
## Modeling Complex Workflows {#modeling-complex-workflows}
47+
48+
This section provides guides to model complex, non-linear CI/CD processes.
49+
50+
### Fan-out Fan-In {#fan-out-fan-in}
51+
52+
<VideoTutorial title="Fan Out - Fan In" src="https://www.youtube.com/embed/HKv-jMkC7T0" />
53+
54+
The Fan-Out Fan-In workflow provides consistency and maximum parallelization. It can be split into 3 stages:
55+
56+
1. **Build stage**: you build your project once
57+
2. **Fan-Out stage**: all your tests fan out from the build stage and run in parallel
58+
3. **Fan-In stage**: once tested, the workflow fans in to a release or deploy stage
59+
60+
![Fan-Out Fan-In](./img/fan-out-fan-in.jpg)
61+
62+
<Tabs groupId="editor-yaml">
63+
<TabItem value="editor" label="Editor" default>
64+
65+
<Steps>
66+
67+
1. Create your build [job](./jobs). Depending on the nature of your project, you can save the built artifact to the [artifact store](./artifacts) or push it to a [Docker registry](./containers/docker)
68+
69+
![Build stage](./img/fan-out-stage1.jpg)
70+
71+
2. Add your test jobs. Set dependencies so all your tests depend on the build job created on Step 1
72+
73+
![Fan-out stage](./img/fan-out-stage2.jpg)
74+
75+
3. Add the release/deploy job. Use dependencies so the new job depends on all your tests. This will make the release/job run only if all tests have passed
76+
77+
![Fan-in stage](./img/fan-out-stage3.jpg)
78+
79+
</Steps>
80+
81+
</TabItem>
82+
<TabItem value="yaml" label="YAML">
83+
84+
You can create a Fan-Out Fan-In workflow by setting the dependencies in your blocks. The Fan-Out stage is achieved by defining `dependencies`. For example:
85+
86+
```yaml
87+
blocks:
88+
- name: "Build"
89+
dependencies: []
90+
...
91+
92+
- name: "Unit tests"
93+
dependencies: ["Build"]
94+
...
95+
96+
- name: "Integration tests"
97+
dependencies: ["Build"]
98+
...
99+
100+
- name: "E2E tests"
101+
dependencies: ["Build"]
102+
...
103+
104+
- name: "Release candidate"
105+
dependencies: ["Integration tests", "Unit tests", "E2E tests"]
106+
...
107+
```
108+
109+
Find below a complex example of a Fan-Out, Fan-In for a Node-based workflow:
110+
111+
```yaml
112+
version: v1.0
113+
name: Continuous Integration Pipeline
114+
agent:
115+
machine:
116+
type: f1-standard-2
117+
os_image: ubuntu2204
118+
blocks:
119+
- name: Build Stage
120+
dependencies: []
121+
task:
122+
jobs:
123+
- name: Build Project
124+
commands:
125+
- 'checkout'
126+
- 'npm run build'
127+
- 'artifact push workflow build/'
128+
- name: End to end tests
129+
dependencies:
130+
- Build Stage
131+
task:
132+
jobs:
133+
- name: Run E2E tests
134+
commands:
135+
- 'checkout'
136+
- 'artifact pull workflow build/'
137+
- 'npm run test:e2e'
138+
- name: Integration Tests
139+
dependencies:
140+
- Build Stage
141+
task:
142+
jobs:
143+
- name: Run integration tests
144+
commands:
145+
- 'checkout'
146+
- 'artifact pull workflow build/'
147+
- 'npm run test:integration'
148+
- name: Unit tests
149+
dependencies:
150+
- Build Stage
151+
task:
152+
jobs:
153+
- name: Run unit tests
154+
commands:
155+
- 'checkout'
156+
- 'artifact pull workflow build/'
157+
- 'npm run test:unit'
158+
- name: Release candidate
159+
dependencies:
160+
- End to end tests
161+
- Integration Tests
162+
- Unit tests
163+
task:
164+
jobs:
165+
- name: Generate RC
166+
commands:
167+
- 'checkout'
168+
- 'artifact pull workflow build/'
169+
- 'npm run package'
170+
```
171+
172+
</TabItem>
173+
</Tabs>
174+
46175
## Workflow triggers
47176
48177
The following events or actions trigger workflows by default:
49178
50179
- Pushing commits into any branch
51180
- Pushing Git tags
52181
- Changing any pipelines
53-
- Manually re-running workflows
182+
- Manually re-running workflows
54183
- Running pipelines using [Tasks](./tasks)
55184
56185
Additionally, you can configure workflows to be triggered by:
@@ -117,12 +246,11 @@ Approving forked pull requests is limited to new comments only and does not work
117246

118247
## How to skip commits {#skip}
119248

120-
If you don't want to start a workflow, type one of the following strings in the commit message.
249+
If you don't want to start a workflow, type one of the following strings in the commit message.
121250

122251
- `[ci skip]`
123252
- `[skip ci]`
124253

125-
126254
For example, this push does not trigger a Semaphore pipeline execution, it is completely ignored:
127255

128256
```shell title="Skipping a commit"
137 KB
Loading
211 KB
Loading
241 KB
Loading
243 KB
Loading

0 commit comments

Comments
 (0)