Skip to content

Commit 1db5bb7

Browse files
committed
Co-execution client Google Cloud Platform Batch v1.
1 parent b60890c commit 1db5bb7

11 files changed

+911
-10
lines changed

docs/containers.rst

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -132,6 +132,22 @@ GA4GH TES
132132

133133
GA4GH TES job execution with Conda dependencies for the tool and no message queue.
134134

135+
Google Cloud Platform Batch
136+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
137+
138+
.. figure:: pulsar_gcp_coexecution_deployment.plantuml.svg
139+
140+
GA4GH TES job execution with a biocontainer for the tool and no message queue.
141+
142+
.. figure:: pulsar_gcp_deployment.plantuml.svg
143+
144+
GA4GH TES job execution with Conda dependencies for the tool and no message queue.
145+
146+
Pulsar job destination options to configure these scenarios:
147+
148+
.. figure:: job_destination_parameters_gcp.png
149+
150+
135151
AWS Batch
136152
~~~~~~~~~~
137153

docs/gen_erd_diagrams.py

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
import os
2+
import sys
3+
4+
import erdantic as erd
5+
6+
sys.path.insert(1, os.path.abspath(os.path.join(os.path.dirname(__file__), os.pardir)))
7+
8+
from pulsar.client.container_job_config import (
9+
GcpJobParams,
10+
)
11+
12+
DOC_SOURCE_DIR = os.path.abspath(os.path.join(os.path.dirname(__file__)))
13+
class_to_diagram = {
14+
GcpJobParams: "job_destination_parameters_gcp",
15+
}
16+
17+
for clazz, diagram_name in class_to_diagram.items():
18+
erd.draw(clazz, out=f"{DOC_SOURCE_DIR}/{diagram_name}.png")
25.4 KB
Loading
Lines changed: 170 additions & 0 deletions
Loading
Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
@startuml
2+
3+
!include plantuml_options.txt
4+
5+
component galaxy as "galaxy" {
6+
7+
}
8+
9+
storage disk as "Object Store" {
10+
11+
}
12+
13+
note as disknote
14+
Disk is unrestricted and does
15+
not need to be shared between
16+
Pulsar and Galaxy.
17+
end note
18+
19+
disk ... disknote
20+
21+
cloud cluster as "Google Cloud Platform" {
22+
queue api as "Google Batch v1 API" {
23+
24+
}
25+
26+
frame pod as "Job" {
27+
28+
component staging as "pulsar Task" {
29+
}
30+
31+
component tool as "biocontainer Task" {
32+
}
33+
34+
}
35+
36+
note as stagingnote
37+
Pulsar Tasks run with a batch_v1.AllocationPolicy.Disk of type "local-ssd"
38+
that is used for Pulsar's staging directory and shared across pulsar stagings and
39+
tool/biocontainer containers.
40+
end note
41+
pod ... stagingnote
42+
}
43+
44+
galaxy --> disk
45+
galaxy --> api : create, delete, get (for status)
46+
api -[dashed]-> pod : [manages]
47+
staging --> galaxy : stage in and out
48+
@enduml

0 commit comments

Comments
 (0)