|
| 1 | +<img src="./1.2/databricks-logo.png" alt="databricks-logo" width="100"/> |
| 2 | + |
| 3 | +# Databricks Extension v2.5.0 |
| 4 | + |
| 5 | +## What's new? |
| 6 | + |
| 7 | +- [Pipeline validation and partial update actions](#validation) |
| 8 | +- [Pipeline events and diagnostics](#events) |
| 9 | +- [Pipeline datasets](#datasets) |
| 10 | +- [Quick run action](#quick-run) |
| 11 | + |
| 12 | +### <a id="validation"></a> Pipeline validation and partial update actions |
| 13 | + |
| 14 | +In previous versions, you could only trigger a [Refresh all](https://docs.databricks.com/en/delta-live-tables/updates.html#how-dlt-updates) pipeline update. Now you can do validation and partial updates: |
| 15 | + |
| 16 | +<img src="./2.5/validation.png" alt="validation action" width="600"/> |
| 17 | + |
| 18 | +The partial update action preloads datasets from past runs, and lets you enter table names or select a full refresh: |
| 19 | + |
| 20 | +<img src="./2.5/partial-update.png" alt="partial update action" width="600"/> |
| 21 | + |
| 22 | +### <a id="events"></a> Pipeline events and diagnostics |
| 23 | + |
| 24 | +During and after a pipeline update, the extension now shows events in the resource explorer UI: |
| 25 | + |
| 26 | +<img src="./2.5/events.png" alt="pipeline events" width="600"/> |
| 27 | + |
| 28 | +Any warnings or errors that have source locations are also displayed in the Visual Studio Code **PROBLEMS** panel and highlighted in the source files: |
| 29 | + |
| 30 | +<img src="./2.5/diagnostics.gif" alt="pipeline diagnostics" width="600"/> |
| 31 | + |
| 32 | +### <a id="datasets"></a> Pipeline datasets |
| 33 | + |
| 34 | +You can now see pipeline datasets and their schemas in the extension resource explorer: |
| 35 | + |
| 36 | +<img src="./2.5/datasets.png" alt="pipeline datasets" width="600"/> |
| 37 | + |
| 38 | +Datasets are gathered from the events of past runs, so make sure to update or validate your pipeline at least once. |
| 39 | + |
| 40 | +### <a id="quick-run"></a> Quick run action |
| 41 | + |
| 42 | +Previously, run and update actions were only available from the extension resource explorer UI. Now you can also use a new `Deploy the bundle and run a resource` command from the Command Palette: |
| 43 | + |
| 44 | +<img src="./2.5/run-command-search.png" alt="pipeline datasets" width="600"/> |
| 45 | + |
| 46 | +By default, you select the resource to execute: |
| 47 | +<img src="./2.5/run-command-ui.png" alt="pipeline datasets" width="600"/> |
| 48 | + |
| 49 | +You can also specify a shortcut to run the command with specific arguments. |
| 50 | + |
| 51 | +Open the keyboard shortcuts JSON using the command `Preferences: Open Keyboard Shortcuts (JSON)`. Add a new entry to the `keybindings` array: |
| 52 | + |
| 53 | +```json |
| 54 | +{ |
| 55 | + // Replace with your desired shortcut |
| 56 | + "key": "ctrl+shift+x", |
| 57 | + "command": "databricks.bundle.deployAndRunFromInput", |
| 58 | + "args": { |
| 59 | + // "jobs" or "pipelines" |
| 60 | + "resourceType": "pipelines", |
| 61 | + // The key of the pipeline or job you want to run, as defined in your asset bundle configuration |
| 62 | + "resourceKey": "my_pipeline_key", |
| 63 | + // Any flags that will be passed to the "databricks bundle run" command |
| 64 | + "args": "--validate-only" |
| 65 | + } |
| 66 | +} |
| 67 | +``` |
0 commit comments