|
1 | 1 | # QuiltSync |
2 | 2 |
|
3 | | -**Desktop client for seamless access to versioned, AI-ready datasets.** |
| 3 | +QuiltSync is a desktop application for syncing versioned Quilt data packages to |
| 4 | +your local machine. It provides local access to Quilt packages stored in S3, |
| 5 | +with support for Windows 10+, macOS 10.14+ (Intel & Apple Silicon), and Linux. |
4 | 6 |
|
5 | | -Visit [quilt.bio/quiltsync](https://quilt.bio/quiltsync/) to download the |
6 | | -latest version. |
| 7 | +## Features |
7 | 8 |
|
8 | | -QuiltSync is a desktop application from Quilt Data that enables scientists, |
9 | | -researchers, and engineers to access, manage, and version large datasets |
10 | | -locally. Install once and seamlessly sync Quilt data packages (versioned, |
11 | | -AI/ML-ready datasets) to your computer across Windows, macOS (Intel & Apple |
12 | | -Silicon), and Linux. |
| 9 | +- Browse and sync packages via graphical interface |
| 10 | +- Selective file sync to manage disk space |
| 11 | +- Version control for data packages |
| 12 | +- Browser-based authentication |
| 13 | +- Auto-generated commit messages (v0.14+) |
13 | 14 |
|
14 | 15 | ## Getting Started |
15 | 16 |
|
16 | | -When viewing packages in the Quilt web catalog, you can open them directly in |
17 | | -QuiltSync: |
| 17 | +### Installation |
18 | 18 |
|
19 | | -1. Navigate to a package in your Quilt catalog |
20 | | -2. Click the "Get Package" button and select "QuiltSync" |
21 | | -3. QuiltSync will open automatically (if installed) and begin syncing the |
22 | | - package |
| 19 | +Download and install QuiltSync from |
| 20 | +[quilt.bio/quiltsync](https://quilt.bio/quiltsync/). |
23 | 21 |
|
24 | | -### Integration with Benchling |
| 22 | +### Opening Packages and Files |
25 | 23 |
|
26 | | -QuiltSync integrates with the [Benchling Webhook](./benchling.md) to provide |
27 | | -seamless access to notebook-linked packages: |
| 24 | +From the Quilt web catalog: |
| 25 | + |
| 26 | +1. Navigate to a package or file |
| 27 | +2. Click "Get Package" or "Get File" |
| 28 | +3. Select "Open in QuiltSync" |
| 29 | + |
| 30 | + |
| 31 | + |
| 32 | +### Authentication |
| 33 | + |
| 34 | +On first use, QuiltSync prompts for authentication via your web browser: |
| 35 | + |
| 36 | +1. QuiltSync opens your browser to the Quilt Catalog login page |
| 37 | +2. Sign in to your catalog |
| 38 | +3. Copy access token to QuiltSync |
28 | 39 |
|
29 | | -- In Benchling's App Canvas, click the "sync" button next to any package |
30 | | -- The package or file will open directly in QuiltSync |
31 | | -- Changes and updates are reflected across both platforms |
| 40 | +The token is tied to your catalog session. No AWS credentials required. |
32 | 41 |
|
33 | | -## Overview |
| 42 | + |
34 | 43 |
|
35 | | -QuiltSync brings the power of Quilt data packages to your desktop, providing |
36 | | -a local sync solution for cloud-stored data. While datasets may live in |
37 | | -remote storage (S3), QuiltSync gives you local access so you can work offline |
38 | | -or interact with datasets as if they're on your machine. |
| 44 | +### Selective Installation |
39 | 45 |
|
40 | | -## Key Features |
| 46 | +When the package is opened, it shows a list of all files (pre-selected for download). |
41 | 47 |
|
42 | | -### Versioned Data Packages |
| 48 | + |
43 | 49 |
|
44 | | -QuiltSync doesn't just pull files—Quilt packages include version control for |
45 | | -data, tracking changes and enabling reproducible workflows. Each package has |
46 | | -a complete version history, allowing you to: |
| 50 | +### Committing Changes |
47 | 51 |
|
48 | | -- Track data changes over time |
49 | | -- Roll back to previous versions |
50 | | -- Ensure reproducible analysis and ML workflows |
51 | | -- Collaborate with confidence that everyone uses the same data version |
| 52 | +After modifying synced files locally, you can commit changes back to Quilt as a |
| 53 | +new package version: |
52 | 54 |
|
53 | | -### Desktop/Local Access |
| 55 | +1. Open the commit page in QuiltSync |
| 56 | +2. Review the auto-generated commit message, which summarizes the changed files |
| 57 | +3. Edit the message if needed |
| 58 | +4. Click **Commit** to create a new revision |
| 59 | +5. Click **Push** to upload that revision and set it as latest |
54 | 60 |
|
55 | | -Even though data lives in cloud storage, QuiltSync provides local sync |
56 | | -capabilities: |
| 61 | + |
57 | 62 |
|
58 | | -- Work offline with synced datasets |
59 | | -- Interact with S3 data as if it's on your local machine |
60 | | -- Reduce latency for data-intensive operations |
61 | | -- Control which packages and versions are synced locally |
| 63 | +### Settings and Troubleshooting |
62 | 64 |
|
63 | | -### AI-Ready Format |
| 65 | +Access settings via the gear icon in the lower right: |
64 | 66 |
|
65 | | -The platform targets researchers, machine learning engineers, and data science |
66 | | -teams who need clean, versioned datasets prepared for AI/ML pipelines: |
| 67 | + |
| 68 | + |
| 69 | +- **Version**: Current version and release notes |
| 70 | +- **Lineage and cache files**: Opens `.quilt/` directory with package metadata |
| 71 | +- **Logs directory**: Application logs for debugging |
| 72 | +- **Reset state**: "RELOAD PAGE" refreshes UI, "RE-LOGIN" clears authentication |
| 73 | + |
| 74 | +If QuiltSync fails to start after an upgrade, use **RE-LOGIN** or clear the |
| 75 | +`.quilt/` cache directory. Older cached manifests in Parquet format are |
| 76 | +automatically re-fetched from remote storage. |
| 77 | + |
| 78 | +### Integration with Benchling |
| 79 | + |
| 80 | +QuiltSync integrates with the [Benchling Webhook](./benchling.md) to provide |
| 81 | +seamless access to Quilt packages from Benchling notebooks. |
67 | 82 |
|
68 | | -- Datasets formatted for machine learning workflows |
69 | | -- Metadata and schema validation |
70 | | -- Integration with data science tools and notebooks |
71 | | -- Support for large-scale data operations |
| 83 | + |
72 | 84 |
|
73 | | -## System Requirements |
| 85 | +When viewing a package in the Benchling App Canvas: |
74 | 86 |
|
75 | | -QuiltSync runs on all major operating systems: |
| 87 | +1. Click the "sync" button next to any package or file |
| 88 | +2. QuiltSync automatically opens with the selected package |
| 89 | +3. Select files to sync locally |
| 90 | +4. Work offline with your data |
76 | 91 |
|
77 | | -- **Operating Systems**: Windows 10+, macOS 10.14+, Linux (modern |
78 | | - distributions) |
79 | | -- **Disk Space**: Varies based on package sizes you plan to sync (consider |
80 | | - storage for large datasets) |
81 | | -- **Network**: Internet connection required for syncing with S3 (bandwidth |
82 | | - considerations for large datasets) |
83 | | -- **AWS Access**: Valid AWS credentials configured for accessing your Quilt |
84 | | - buckets |
| 92 | +This integration allows scientists to move from notebook entries to local |
| 93 | +datasets without leaving their Benchling workflow. For more details, see |
| 94 | +[Benchling App Canvas](./benchling.md#benchling-app-canvas). |
0 commit comments