Skip to content

Commit 789e719

Browse files
committed
Added documentation for job multiple task orchestration
1 parent 653b5e7 commit 789e719

File tree

2 files changed

+43
-0
lines changed

2 files changed

+43
-0
lines changed

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22

33
## 0.3.9
44

5+
* Added initial support for multiple task orchestration in `databricks_job` [#853](https://github.com/databrickslabs/terraform-provider-databricks/pull/853)
56
* Fixed provider crash for new terraform states related to bug [#813](https://github.com/hashicorp/terraform-plugin-sdk/issues/813) in Terraform SDK v2.8.0 ([#854](https://github.com/databrickslabs/terraform-provider-databricks/issues/854))
67

78
Updated dependency versions:

docs/resources/job.md

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,48 @@ output "job_url" {
4747
}
4848
```
4949

50+
## Jobs with Multiple Tasks
51+
52+
-> **Note** In terraform configuration, you must define tasks in alphabetical order of their `task_key` arguments, so that you get consistent and readable diff. Whenever tasks are added or removed, or `task_key` is renamed, you'll observe a change in the majority of tasks. It's related to the fact that the current version of the provider treats `task` blocks as an ordered list. Alternatively, `task` block could have been an unordered set, though end-users would see the entire block replaced upon a change in single property of the task.
53+
54+
It is possible to create jobs with multiple tasks using the `task` blocks:
55+
56+
```hcl
57+
resource "databricks_job" "this" {
58+
name = "Job with multiple tasks"
59+
60+
task {
61+
task_key = "a"
62+
63+
new_cluster {
64+
num_workers = 1
65+
spark_version = data.databricks_spark_version.latest.id
66+
node_type_id = data.databricks_node_type.smallest.id
67+
}
68+
69+
notebook_task {
70+
notebook_path = databricks_notebook.this.path
71+
}
72+
}
73+
74+
task {
75+
task_key = "b"
76+
77+
depends_on {
78+
task_key = "a"
79+
}
80+
81+
existing_cluster_id = databricks_cluster.shared.id
82+
83+
spark_jar_task {
84+
main_class_name = "com.acme.data.Main"
85+
}
86+
}
87+
}
88+
```
89+
90+
Every `task` block can have almos all available arguments with the addition of `task_key` attribute and `depends_on` blocks to define cross-task dependencies.
91+
5092
## Argument Reference
5193

5294
The following arguments are required:

0 commit comments

Comments
 (0)