[ACIX-1225] Create `dda env dev fs import` and `export` commands #233

Ishirui · 2025-12-23T15:54:34Z

Create two commands, dda env dev fs import and dda env dev fs export that allow for easily moving files and directories in and out of dev envs.

This is useful as an addition to standard bind mounts, because people might not think of bind-mounting something into the container upon creation. Such a feature was heavily requested by our users.

The feature was implemented by making the necessary changes to the dev env interface to make sure that such a feature will also be supported by other dev env types (remote VMs...). As much logic as possible was made agnostic to the type of dev env in question (everything in src/dda/env/dev/fs.py)

…nd stubs

…ort_from_dir` in dev envs

…ntainer

Copilot

Pull request overview

This PR implements dda env dev fs import and dda env dev fs export commands that enable easy file transfer between the host filesystem and development environments. The feature is designed to be extensible to different environment types (not just Linux containers) by implementing the functionality through the dev env interface.

Key Changes:

Added abstract export_files and import_files methods to the dev environment interface
Implemented core file operation logic in src/dda/env/dev/fs.py that's agnostic to environment type
Created CLI commands for export, import, and an internal localimport command for use inside containers

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
`src/dda/env/dev/interface.py`	Added abstract methods `export_files` and `import_files` to the dev environment interface
`src/dda/env/dev/fs.py`	New module containing environment-agnostic file operation logic (`import_from_dir`, `determine_final_copy_target`, `handle_overwrite`)
`src/dda/env/dev/types/linux_container.py`	Implemented `export_files` and `import_files` methods for Linux containers using docker cp and shared directories; added per-instance shared directory mount
`src/dda/utils/fs.py`	Enhanced `temp_directory` function to accept optional `dir` parameter for controlling where temporary directories are created
`src/dda/cli/env/dev/fs/__init__.py`	New CLI group for filesystem operations
`src/dda/cli/env/dev/fs/export/__init__.py`	CLI command for exporting files from dev environments to host
`src/dda/cli/env/dev/fs/import/__init__.py`	CLI command for importing files from host to dev environments
`src/dda/cli/env/dev/fs/localimport/__init__.py`	Internal CLI command (hidden) used inside dev environments to perform the actual file import
`tests/env/dev/test_fs.py`	Comprehensive unit tests for the file operation logic in `fs.py`
`tests/env/dev/test_interface.py`	Updated interface test stub to include new methods
`tests/env/dev/types/test_linux_container.py`	Updated tests for shared directory path changes and added tests for export/import orchestration
`tests/env/dev/fixtures/fs_tests/*`	Test fixture files for verifying file operations

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/dda/env/dev/types/linux_container.py

src/dda/env/dev/interface.py

src/dda/cli/env/dev/fs/export/__init__.py

src/dda/env/dev/fs.py

src/dda/cli/env/dev/fs/localimport/__init__.py

chouetz · 2026-01-05T11:07:31Z

src/dda/cli/env/dev/fs/import/__init__.py

+    )
+    status = env.status()
+
+    possible_states = {EnvironmentState.STARTED}


Will it be possible to have more states in the future? Otherwise you can maybe simplify the code

Yes, similar to the same block in export, this might end up being different for every environment type. We could maybe imagine modifying a VM's filesystem even when it is stopped by directly editing the underlying volumes for example.

chouetz · 2026-01-05T11:09:15Z

src/dda/cli/env/dev/fs/export/__init__.py

+    status = env.status()
+
+    # TODO: This might end up depending on the environment type.
+    # For `linux-container` though, `docker cp` also works on stopped containers.


Does this comment mean the command only works partially (ie just for linux-container environment)?

Well for the moment that's the only environment type we have, so yes it's only implemented for linux-container as of yet. But I've added abstract methods in the interface definition so every type we implement in the feature will need to implement the required methods as well.

ofek

Thanks!

ofek · 2026-01-05T17:35:09Z

src/dda/cli/env/dev/fs/export/__init__.py

@@ -0,0 +1,73 @@
+# SPDX-FileCopyrightText: 2024-present Datadog, Inc. <[email protected]>


Let's make sure the years match when new files were introduced.

src/dda/cli/env/dev/fs/import/__init__.py

ofek · 2026-01-06T07:18:25Z

src/dda/cli/env/dev/fs/localimport/__init__.py

+    mkpath: bool,
+) -> None:
+    """
+    Internal command used to call import_from_dir in dev envs.


Why is this specific function required when we can make any subprocess call within environments?

Remove this ; discussed offline - just use cp -r

ofek · 2026-01-06T07:48:08Z

src/dda/env/dev/types/linux_container.py

+        from dda.utils.fs import temp_directory
+
+        # 1. Create a temporary directory on the host filesystem
+        with temp_directory() as wd:


I think that we should use the shared directory for communication in all circumstances just like in the import_files method and then have the host move the staged source to the destination. The only situation in which this wouldn't be as desirable would be copying many small files but the main use case here is large build artifacts.

Tested offline: using the shared dir is way faster (in both directions) than docker cp (!), at least for single large files (since we'll mostly be exfiltrating build artifacts this is the most common usecase)

-> Use the shared dir for both methods

Ishirui · 2026-01-06T16:30:47Z

src/dda/cli/env/dev/fs/import/__init__.py

+@dynamic_command(short_help="""Import files and directories into a developer environment""")
+@option_env_type()
+@click.option("--id", "instance", default="default", help="Unique identifier for the environment")
+@click.argument("sources", nargs=-1, required=True, type=click.Path(exists=True, resolve_path=True, path_type=Path))


Only allow one source: globs don't work so it would be weird

Ishirui · 2026-01-06T17:50:15Z

src/dda/env/dev/fs.py

+    dest.unlink()
+
+
+def import_from_dir(source_dir: Path, destination_spec: Path, *, recursive: bool, force: bool, mkpath: bool) -> None:


We can probably remove this entire file, as we will just rely on the semantics of cp and mv - note cp does overwrite files if you ask it to.

Ishirui · 2026-01-06T17:50:27Z

src/dda/env/dev/interface.py

+        raise NotImplementedError
+
+    @abstractmethod
+    def import_files(


Keep this abstraction though, it's good

Ishirui force-pushed the pierrelouis.veyrenc/ACIX-1225-env-dev-fs-commands branch 2 times, most recently from bffa189 to b055041 Compare December 29, 2025 11:21

Ishirui added 3 commits December 29, 2025 12:21

feat(env): Create fs command group with export and import comma…

0d4165e

…nd stubs

feat(fs): Implement skeleton for export and import commands

f398217

fix(fs): Pass non-host paths as str

f6638d4

Ishirui force-pushed the pierrelouis.veyrenc/ACIX-1225-env-dev-fs-commands branch from b055041 to dbe0c30 Compare December 29, 2025 11:21

Ishirui added 2 commits December 29, 2025 16:25

feat(env): Implement LinuxContainer.export_files

3cd4d22

refactor(env): Move host filesystem import logic to fs.py

95901d4

Ishirui force-pushed the pierrelouis.veyrenc/ACIX-1225-env-dev-fs-commands branch from dbe0c30 to 95901d4 Compare January 2, 2026 14:26

Ishirui added 4 commits January 2, 2026 15:47

feat(env): Implement hidden localimport command to easily call `imp…

08d4b2e

…ort_from_dir` in dev envs

feat(fs): Allow passing dir to temp_directory

ce5915e

feat(env): Implement LinuxContainer.import_files

d12ba33

fix(linux_container): Actually mount the shared directory into the co…

78b4128

…ntainer

Ishirui changed the title ~~Pierrelouis.veyrenc/ACIX-1225-env-dev-fs-commands~~ [ACIX-1225] Create dda env dev fs import and export commands Jan 2, 2026

fix(env): Small fixes for import command

721dada

Ishirui force-pushed the pierrelouis.veyrenc/ACIX-1225-env-dev-fs-commands branch from eb939cb to 721dada Compare January 2, 2026 17:42

Ishirui marked this pull request as ready for review January 2, 2026 17:42

Ishirui requested a review from a team as a code owner January 2, 2026 17:42

Copilot AI review requested due to automatic review settings January 2, 2026 17:42

Copilot started reviewing on behalf of Ishirui January 2, 2026 17:43 View session

Copilot AI reviewed Jan 2, 2026

View reviewed changes

Ishirui added 2 commits January 5, 2026 11:04

fix(fs): Make sure we docker cp into the temporary directory

82d700a

refactor: Copilot-suggested review changes

365fdad

chouetz approved these changes Jan 5, 2026

View reviewed changes

ofek reviewed Jan 6, 2026

View reviewed changes

Ishirui commented Jan 6, 2026

View reviewed changes

		@@ -0,0 +1,73 @@
		# SPDX-FileCopyrightText: 2024-present Datadog, Inc. <[email protected]>

		dest.unlink()


		def import_from_dir(source_dir: Path, destination_spec: Path, *, recursive: bool, force: bool, mkpath: bool) -> None:

[ACIX-1225] Create dda env dev fs import and export commands #233

Are you sure you want to change the base?

[ACIX-1225] Create dda env dev fs import and export commands #233

Uh oh!

Conversation

Ishirui commented Dec 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ofek left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

[ACIX-1225] Create `dda env dev fs import` and `export` commands #233

[ACIX-1225] Create `dda env dev fs import` and `export` commands #233

Ishirui commented Dec 23, 2025 •

edited

Loading