|
| 1 | +# Quilt: Scientific Data Management on AWS |
| 2 | + |
| 3 | +[](https://docs.quilt.bio/) |
| 4 | +[](https://riverside.com/webinar/registration/eyJzbHVnIjoic2ltb24ta29obnN0YW1tcy1zdHVkaW8iLCJldmVudElkIjoiNjk5Y2M5MDI4YjVkM2Y2MjFjYTk5MzgzIiwicHJvamVjdElkIjoiNjk5Y2M5MDI1OTE5NDU0YmNlOWEzZDVmIn0=) |
| 5 | +[](https://www.quilt.bio/meetings/simon-kohnstamm/quilt-introduction) |
| 6 | + |
| 7 | +Quilt is a **Scientific Data Management Platform on AWS** that helps teams and |
| 8 | +AI find, trust, and reuse data through deeply versioned, context-rich data |
| 9 | +packages. |
| 10 | + |
| 11 | +Most scientific organizations do not struggle to generate data. They struggle |
| 12 | +to keep it usable over time. As teams, tools, and workflows evolve, context |
| 13 | +gets lost, scientists cannot find what they need, data teams get pulled into |
| 14 | +manual support, and AI projects slow down because data lacks structure, |
| 15 | +lineage, and trust. |
| 16 | + |
| 17 | +Quilt solves this by turning cloud data into durable, searchable, reusable |
| 18 | +packages. Each package captures data plus metadata, documentation, lineage, and |
| 19 | +version history needed for confident reuse. Built on AWS, Quilt works with data |
| 20 | +in place, so organizations can improve data management without disruptive |
| 21 | +migrations or rigid workflows. |
| 22 | + |
| 23 | +## Open Source and Enterprise |
| 24 | + |
| 25 | +Quilt includes both open-source software and an enterprise platform deployment model. |
| 26 | + |
| 27 | +- **Open source (this repository):** |
| 28 | + - Python SDK and CLI for creating Quilt packages, installing packages |
| 29 | + locally, and uploading packages to S3 using your AWS credentials |
| 30 | + - Package versioning, reproducibility, and documentation workflows that fit |
| 31 | + data science and bioinformatics workloads |
| 32 | + - Does **not** provide the full hosted search and visualization experience |
| 33 | + for package discovery and collaboration |
| 34 | +- **Enterprise platform:** |
| 35 | + - Dedicated AWS-hosted Quilt platform for teams to search, share, and |
| 36 | + visualize Quilt packages |
| 37 | + - Multi-user collaboration and governance features on top of open-source |
| 38 | + package workflows |
| 39 | + |
| 40 | +Start with: |
| 41 | + |
| 42 | +- Open-source docs: [`docs/README.md`](docs/README.md) |
| 43 | +- Enterprise administration docs: [`docs/technical-reference.md`](docs/technical-reference.md) |
| 44 | +- Contributor guide: [`docs/CONTRIBUTING.md`](docs/CONTRIBUTING.md) |
| 45 | + |
| 46 | +## Repository Map |
| 47 | + |
| 48 | +Use this to find where each major part of Quilt lives. |
| 49 | + |
| 50 | +| Path | What it contains | |
| 51 | +| --- | --- | |
| 52 | +| `api/python` | `quilt3` Python SDK, CLI, and Python tests | |
| 53 | +| `catalog` | Quilt web catalog frontend (TypeScript/JavaScript) | |
| 54 | +| `lambdas` | AWS Lambda services (indexing, previews, events, etc.) | |
| 55 | +| `shared` | Shared schemas and cross-component assets | |
| 56 | +| `py-shared` | Shared Python utilities used by services | |
| 57 | +| `s3-proxy` | S3 proxy service components | |
| 58 | +| `docs` | Product, platform, API, and contributor documentation | |
| 59 | +| `gendocs` | API documentation generation tooling | |
| 60 | +| `testdocs` | Documentation codeblock validation tools | |
| 61 | + |
| 62 | +## Quick Start for Contributors |
| 63 | + |
| 64 | +1. Clone and create a branch: |
| 65 | + |
| 66 | + ```bash |
| 67 | + git clone https://github.com/quiltdata/quilt |
| 68 | + cd quilt |
| 69 | + git checkout -b my-change |
| 70 | + ``` |
| 71 | + |
| 72 | +2. Install Python task runner: |
| 73 | + |
| 74 | + ```bash |
| 75 | + # macOS/Linux |
| 76 | + curl -LsSf https://astral.sh/uv/install.sh | sh |
| 77 | + ``` |
| 78 | + |
| 79 | +3. Run Python tests: |
| 80 | + |
| 81 | + ```bash |
| 82 | + cd api/python |
| 83 | + uv run poe test |
| 84 | + ``` |
| 85 | + |
| 86 | +4. Run catalog locally: |
| 87 | + |
| 88 | + ```bash |
| 89 | + cd catalog |
| 90 | + npm install |
| 91 | + npm start |
| 92 | + ``` |
| 93 | + |
| 94 | +For full setup, testing, and release steps, see [`docs/CONTRIBUTING.md`](docs/CONTRIBUTING.md). |
| 95 | + |
| 96 | +## Learn More |
| 97 | + |
| 98 | +- Product docs: [docs.quilt.bio](https://docs.quilt.bio/) |
| 99 | +- Open data demo catalog: [open.quiltdata.com](https://open.quiltdata.com/) |
| 100 | +- Case studies and customer stories: [quilt.bio/case-studies](https://www.quilt.bio/case-studies) |
| 101 | +- Slack community: [Join Quilt Slack][quilt-slack] |
| 102 | + |
| 103 | +[quilt-slack]: https://slack.quilt.bio |
0 commit comments