Skip to content

Commit 4193374

Browse files
andrewjcgfacebook-github-bot
authored andcommitted
Add a library to sync conda environments (#628)
Summary: Pull Request resolved: #628 X-link: #628 This adds a crate to do rsync-style syncing of a source conda env to a destination, as per: ``` $ rsync --archive --update --no-dirtimes --delete --delete-excluded ... ``` As `rsync` (and this crate) uses mtimes for syncing, it'll work even with the differences in prefixes between source and destination conda envs. The notable differences with rsync are: 1) In current flows, when users "activate" their conda env locally, prefixes in the env will be quickly re-written, changing mtimes to be ahead of the destination (e.g. on MAST, the conda env is unpacked into it's expected prefix location). This crate uses the `pack-meta/history.jsonl` file to detect prefix update windows and use that to filter out these sprurious mtime updates. 2) Files that are copied perform in-place prefix replacement (TODO: not finished yet). Reviewed By: highker Differential Revision: D78424481 fbshipit-source-id: 0a433baee8aff90b4b05091d9b3188dd10655b44
1 parent 3db8984 commit 4193374

File tree

6 files changed

+2703
-0
lines changed

6 files changed

+2703
-0
lines changed

monarch_conda/Cargo.toml

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# @generated by autocargo from //monarch/monarch_conda:[conda-sync-cli,monarch_conda]
2+
3+
[package]
4+
name = "monarch_conda"
5+
version = "0.0.0"
6+
authors = ["Meta"]
7+
edition = "2021"
8+
license = "BSD-3-Clause"
9+
10+
[[bin]]
11+
name = "conda_sync_cli"
12+
path = "src/main.rs"
13+
14+
[dependencies]
15+
anyhow = "1.0.98"
16+
async-tempfile = "0.7.0"
17+
bincode = "1.3.3"
18+
clap = { version = "4.5.42", features = ["derive", "env", "string", "unicode", "wrap_help"] }
19+
dashmap = { version = "5.5.3", features = ["rayon", "serde"] }
20+
digest = "0.10"
21+
filetime = "0.2.25"
22+
futures = { version = "0.3.31", features = ["async-await", "compat"] }
23+
globset = { version = "0.4.13", features = ["serde1"] }
24+
ignore = "0.4"
25+
itertools = "0.14.0"
26+
memchr = "2.7.5"
27+
memmap2 = "0.9.5"
28+
rattler_conda_types = "0.28.3"
29+
serde = { version = "1.0.219", features = ["derive", "rc"] }
30+
serde_json = { version = "1.0.140", features = ["alloc", "float_roundtrip", "raw_value", "unbounded_depth"] }
31+
sha2 = "0.10.6"
32+
tokio = { version = "1.46.1", features = ["full", "test-util", "tracing"] }
33+
tokio-util = { version = "0.7.15", features = ["full"] }
34+
walkdir = "2.3"
35+
36+
[dev-dependencies]
37+
tempfile = "3.15"

0 commit comments

Comments
 (0)