-
Notifications
You must be signed in to change notification settings - Fork 37
feat: Introduce deploy command
#180
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
aviatco
merged 54 commits into
microsoft:main
from
aviatco:dev/aviatcohen/cli-cicd-intergation-main
Mar 10, 2026
Merged
Changes from 42 commits
Commits
Show all changes
54 commits
Select commit
Hold shift + click to select a range
b32c344
Introduce deploy command
a5f3d49
add changie
c319fb6
Resolve PR commants
a727e79
Create MSAL bridge
3c3bf67
remove metavar
c472fac
align context with other commands - will handle it in another PR
176c090
Merge pull request #2 from aviatco/deploy-command-interface
aviatco f9cb1f7
Merge branch 'dev/aviatcohen/cli-cicd-intergation-main' into dv/aviatβ¦
1a9bd67
call cicd deploy_with_config
4c53e4e
resolve PR cooments
508b7ca
Update src/fabric_cli/core/fab_auth.py
aviatco 06c2dd8
resolve commants
d844bba
remove handling of expires_on when converting msal token to azure ideβ¦
170b45a
make sure expires_on is int
a02687e
Merge pull request #3 from aviatco/dev/aviatcohen/msal-bridge
aviatco 5972eb1
support deploy command
73bbdbd
Merge branch 'dev/aviatcohen/msal-bridge' into dev/aviatcohen/deploy-β¦
e9c0fc0
using expires_in if expires_on is not avilable
d1de4ec
Merge branch 'dev/aviatcohen/cli-cicd-intergation-main' into dev/aviaβ¦
870e923
comment to disable live recording mode
4fb7b7b
change --deploy_with_config flag to config; fix PR comments
3d733cd
create deploy_setup fixture and update tests
08d7d8d
add tests
9991a00
Update src/fabric_cli/commands/fs/deploy/fab_fs_deploy_config_file.py
aviatco d3e8dba
update cicd feature flags and debug_enabled
69ef52c
Merge branch 'dev/aviatcohen/deploy-with-config' of https://github.coβ¦
d21ff0d
update cicd feature flags and debug_enabled
ca31a7b
Merge pull request #4 from aviatco/dev/aviatcohen/deploy-with-config
aviatco a2a3243
add changie
785ee13
align seccess msg with cicd updates
58acead
define VALID_DEFATULT_SCOPES at module level
a4c8c96
Merge branch 'main' into dev/aviatcohen/cli-cicd-intergation-main
aviatco c13be3d
Fix type check tests
1511aba
fix type check error for expires_on
de8660f
remove python 3.13 from test build just for testing - will be revertgβ¦
8d55167
resolve backslash issue in prompt message
a764267
fix test_get_access_token_token_error
128702c
Apply suggestion from @aviatco
aviatco 75c35f1
trying to fix interactive_cli tests
264bdbc
Merge branch 'dev/aviatcohen/cli-cicd-intergation-main' of https://giβ¦
39aa30e
deploy documentation
e6cf9d5
Merge branch 'main' into dev/aviatcohen/cli-cicd-intergation-main
aviatco 9467b6b
bump cicd version
d4f5f52
remove deploy docs
c1082b3
Merge branch 'dev/aviatcohen/cli-cicd-intergation-main' of https://giβ¦
1b8b174
revert - remove python 3.13 support
64838f0
fix deploy tests
b7bdfde
mock acquire_token method
0a8def4
mock acquire_token
7fb1e55
new decording for test_deploy_single_item_success
dc7fd75
revert scope class fixture
23f06f1
revert setup_default_format scopr class in mock_fab_set_state_config
e4cf5b2
revert setup_default_format scope class in setup_default_format
17e4a67
recorde fail tests
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| kind: added | ||
| body: Introduces the new deploy command that integrates with the fabric-cicd library, enabling users to deploy multiple Fabric items in a single | ||
| time: 2026-03-05T09:17:28.270036405Z | ||
| custom: | ||
| Author: aviatco | ||
| AuthorLink: https://github.com/aviatco |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,224 @@ | ||
| # `deploy` Command | ||
|
|
||
| Deploy a Fabric workspace from local source content into a target Microsoft Fabric workspace. | ||
|
|
||
| --- | ||
|
|
||
| ## Description | ||
|
|
||
| The `deploy` command imports workspace content from a local source into a target Fabric workspace. | ||
aviatco marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| You can deploy: | ||
| - an entire workspace | ||
aviatco marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| - specific folders | ||
| - specific items | ||
|
|
||
| The scope and behavior of the deployment are defined entirely by the deployment configuration file. | ||
|
|
||
aviatco marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| During execution, the command: | ||
| - publishes items found in the source into the target workspace | ||
aviatco marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| - resolves dependencies between items automatically when logical IDs are present | ||
| - removes items from the target workspace that no longer exist in the source (unpublish) | ||
|
|
||
aviatco marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| By default, **both publish and unpublish operations are enabled and executed**. | ||
| To disable either operation for a specific environment, it must be explicitly skipped in the configuration file. | ||
|
|
||
| The deployment is executed directly against Fabric workspaces using Fabric APIs. | ||
aviatco marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| --- | ||
|
|
||
| ## Usage | ||
|
|
||
| ```bash | ||
| fab deploy --config <config_file> [--target_env <environment>] [--params <parameters>] [--force] | ||
| ``` | ||
|
|
||
| --- | ||
|
|
||
| ## Options | ||
|
|
||
| | Option | Description | | ||
| |------|-------------| | ||
| | `--config <file>` | Path to the deployment configuration file. **Required**. | | ||
| | `--target_env, -tenv <env>` | Environment name used to select environment-specific settings from the configuration file. | | ||
aviatco marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| | `--params, -P <params>` | JSON-formatted parameters provided to the deployment process at runtime. | | ||
| | `--force, -f` | Run the deployment without interactive confirmation prompts. | | ||
aviatco marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| | `--help` | Show detailed help for the `deploy` command. | | ||
|
|
||
| --- | ||
|
|
||
| ## Configuration Behavior | ||
|
|
||
| - The configuration file controls what is published and unpublished. | ||
| - Publish and unpublish operations are enabled by default. | ||
| - Skipping publish or unpublish must be explicitly defined per environment. | ||
| - Environment selection is resolved only when environment mappings are present in the configuration. | ||
| - If `--target_env` is not specified, the configuration must not contain environment-specific mappings. | ||
|
|
||
| --- | ||
|
|
||
| ## Configuration File | ||
|
|
||
| ### Minimal Configuration (No Environments) | ||
aviatco marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ```yaml | ||
| core: | ||
| workspace_id: "12345678-1234-1234-1234-123456789abc" | ||
| repository_directory: "." | ||
| ``` | ||
|
|
||
| --- | ||
|
|
||
| ### Configuration Fields | ||
|
|
||
| | Field | Description | Mandatory | | ||
| |------|------------|-----------| | ||
| | `core.workspace_id` | Target workspace ID (GUID). Takes precedence over `workspace`. | β Yes | | ||
| | `core.workspace` | Target workspace name. Alternative to `workspace_id`. | β No | | ||
| | `core.repository_directory` | Path to local directory containing workspace content. | β Yes | | ||
aviatco marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| | `core.item_types_in_scope` | List of item types included in deployment. | β No | | ||
| | `core.parameter` | Path to parameter file. | β No | | ||
| | `publish` | Controls publishing behavior. | β No | | ||
| | `unpublish` | Controls unpublishing behavior. | β No | | ||
| | `features` | Feature flags. | β No | | ||
aviatco marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| | `constants` | API constant overrides. | β No | | ||
aviatco marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| Relative paths are resolved relative to the configuration file location. | ||
|
|
||
aviatco marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| --- | ||
|
|
||
| ## Publish and Unpublish Settings | ||
|
|
||
| ### Publish | ||
|
|
||
| ```yaml | ||
| publish: | ||
| exclude_regex: "^DONT_DEPLOY.*" # Excludes items matching this pattern from publishing | ||
| folder_exclude_regex: "^legacy/" # Excludes items under matching folders from publishing | ||
aviatco marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| items_to_include: # Publishes only the specified items (requires feature flags) | ||
aviatco marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| - "MainNotebook.Notebook" | ||
| skip: # Skips publishing per environment | ||
| dev: true | ||
| test: false | ||
| prod: false | ||
| ``` | ||
|
|
||
| ### Unpublish | ||
|
|
||
| ```yaml | ||
| unpublish: | ||
| exclude_regex: "^DEBUG.*" # Prevents matching items from being removed | ||
| items_to_include: # Unpublishes only the specified items (requires feature flags) | ||
| - "OldPipeline.DataPipeline" | ||
| skip: # Skips unpublishing per environment | ||
| dev: true | ||
| test: false | ||
| prod: false | ||
| ``` | ||
|
|
||
| > **Note** | ||
| > While selective deployment is supported, it is not recommended due to potential issues with dependency management. Use selective deployment options with caution. | ||
|
|
||
| --- | ||
|
|
||
| ## Parameter File | ||
|
|
||
| The parameter file enables environment-specific transformation of deployed content. | ||
aviatco marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| It can be used to: | ||
| - replace workspace and item identifiers | ||
| - replace connection strings and URLs | ||
| - update embedded values inside item definitions | ||
| - parameterize environment-specific configuration values | ||
|
|
||
| The parameter file is optional and is applied only when specified in the configuration. | ||
|
|
||
| When items were exported or created using Fabric CLI (and not through Git integration), dependencies must be explicitly defined in the parameter file to ensure correct resolution during deployment. | ||
|
|
||
aviatco marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| ### Example | ||
|
|
||
| ```yaml | ||
| find_replace: | ||
| - find_value: "dev-connection-string" | ||
| replace_value: | ||
| dev: "dev-connection-string" | ||
| test: "test-connection-string" | ||
| prod: "prod-connection-string" | ||
| ``` | ||
|
|
||
| ### Supported Variables | ||
|
|
||
| - `$workspace.$id` β resolves to the target workspace ID | ||
| - `$items.<ItemType>.<ItemName>.$id` β resolves to the deployed item ID | ||
|
|
||
| Parameterization behavior follows the documented model described here: | ||
|
|
||
| https://microsoft.github.io/fabric-cicd/0.1.3/how_to/parameterization/add | ||
aviatco marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| --- | ||
|
|
||
| ## Examples | ||
|
|
||
| ### Basic Deployment | ||
|
|
||
| ```bash | ||
| fab deploy --config config.yml | ||
| ``` | ||
|
|
||
| ### Deployment to a Specific Environment | ||
|
|
||
| ```bash | ||
| fab deploy --config config.yml --target_env prod | ||
| ``` | ||
|
|
||
| ### Deployment with Runtime Parameters | ||
|
|
||
| ```bash | ||
| fab deploy --config config.yml --target_env test -P '{"core":{"item_types_in_scope":["Notebook"]}}' | ||
| ``` | ||
|
|
||
| --- | ||
|
|
||
| ## Preparing Source Content | ||
|
|
||
| ### Using Git Integration | ||
|
|
||
| If the workspace is connected to Git, clone the repository locally: | ||
|
|
||
| ```bash | ||
| git clone https://github.com/org/fabric-workspace.git | ||
| ``` | ||
|
|
||
| Use the cloned repository as the deployment source. | ||
|
|
||
| --- | ||
|
|
||
| ### Using `fab export` | ||
|
|
||
| `fab export` exports one item at a time and does not include logical IDs. | ||
|
|
||
| ```bash | ||
| fab export --output ./path-to-ws MyDataPipeline.DataPipeline --format .py | ||
| ``` | ||
|
|
||
| To export Notebook items use the `--format .py` option. | ||
|
|
||
| ```bash | ||
| fab export --output ./path-to-ws MyNotebook.Notebook --format .py | ||
| ``` | ||
|
|
||
| --- | ||
|
|
||
| ## Notes | ||
|
|
||
| - `fab deploy` does not require Git integration; however, using Git integration is strongly recommended to ensure dependencies. | ||
aviatco marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| - The deploy command supports content sourced from Git repositories, locally exported items, and items created locally. | ||
aviatco marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| - Specifying a target environment is optional. If --target_env is provided, only values for that environment are applied. If environmentβspecific values exist in the configuration, --target_env is required. | ||
aviatco marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| - Parameter files are optional and apply to all deployment scenarios. | ||
aviatco marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| --- | ||
|
|
||
| ## Related Commands | ||
|
|
||
| - `export` β Export individual items from a workspace | ||
aviatco marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| - `import` β Import individual items into a workspace | ||
aviatco marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
56 changes: 56 additions & 0 deletions
56
src/fabric_cli/commands/fs/deploy/fab_fs_deploy_config_file.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| # Copyright (c) Microsoft Corporation. | ||
| # Licensed under the MIT License. | ||
|
|
||
| import json | ||
aviatco marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| from argparse import Namespace | ||
|
|
||
| from fabric_cicd import append_feature_flag, configure_external_file_logging, deploy_with_config, disable_file_logging # type: ignore | ||
|
|
||
| from fabric_cli.core import fab_constant, fab_state_config | ||
| from fabric_cli.core import fab_logger | ||
| from fabric_cli.core.fab_exceptions import FabricCLIError | ||
| from fabric_cli.core.fab_msal_bridge import create_fabric_token_credential | ||
| from fabric_cli.utils import fab_ui | ||
| from fabric_cli.utils.fab_util import get_dict_from_params | ||
|
|
||
|
|
||
| def deploy_with_config_file(args: Namespace) -> None: | ||
| """deploy fabric items to a workspace using a configuration file and target environment - delegates to CICD library.""" | ||
|
|
||
| try: | ||
| if fab_state_config.get_config(fab_constant.FAB_DEBUG_ENABLED) == "true": | ||
| cli_logger = fab_logger.get_logger() | ||
| # configure file logging for CICD library to use the same file handler as the CLI | ||
| configure_external_file_logging(cli_logger) | ||
| else: | ||
| # prevent creation of a log file for fabric-cicd logs when debug mode is disabled | ||
| disable_file_logging() | ||
aviatco marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| # feature flags to avoid printing identity info in logs | ||
| append_feature_flag("disable_print_identity") | ||
|
|
||
| deploy_config_file = args.config | ||
| deploy_parameters = get_dict_from_params(args.params, max_depth=1) | ||
| for param in deploy_parameters: | ||
| if isinstance(deploy_parameters[param], str): | ||
| try: | ||
| deploy_parameters[param] = json.loads( | ||
| deploy_parameters[param]) | ||
| except json.JSONDecodeError: | ||
| # If it's not a valid JSON string, keep it as is | ||
| pass | ||
| result = deploy_with_config( | ||
| config_file_path=deploy_config_file, | ||
| environment=args.target_env, | ||
| token_credential=create_fabric_token_credential(), # MSAL bridge TokenCredential | ||
| **deploy_parameters | ||
| ) | ||
|
|
||
| if result: | ||
| fab_ui.print_output_format( | ||
| args, message=result.message) | ||
|
|
||
| except Exception as e: | ||
| raise FabricCLIError( | ||
| f"Deployment failed: {str(e)}", | ||
| fab_constant.ERROR_IN_DEPLOYMENT) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| # Copyright (c) Microsoft Corporation. | ||
| # Licensed under the MIT License. | ||
|
|
||
| from argparse import Namespace | ||
|
|
||
| from fabric_cli.commands.fs.deploy.fab_fs_deploy_config_file import deploy_with_config_file | ||
| from fabric_cli.utils import fab_ui | ||
|
|
||
|
|
||
| def exec_command(args: Namespace) -> None: | ||
| """deploy fabric items to a workspace using a configuration file and target environment - CICD flow.""" | ||
| target_env_msg = 'without a target environment' if args.target_env == 'N/A' else f"to target environment '{args.target_env}'" | ||
| if args.force or fab_ui.prompt_confirm(f"Are you sure you want to deploy {target_env_msg} using the specified configuration file?"): | ||
| deploy_with_config_file(args) |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.