|
1 | 1 |
|
2 | 2 | # AppFlowy-Collab |
3 | 3 |
|
4 | | -`AppFlowy-Collab` is a project that aims to support the collaborative features of AppFlowy. The workspace now centers on |
5 | | -the `collab` crate, which houses database, document, folder, importer, plugin, and user functionality under a single |
6 | | -module tree. |
| 4 | +AppFlowy-Collab is the collaborative data layer that powers AppFlowy clients. It exposes `collab`, a Rust crate built on |
| 5 | +top of [yrs](https://docs.rs/yrs/latest/yrs/), packaging CRDT primitives, persistence helpers, and domain objects for |
| 6 | +documents, databases, folders, importers, plugins, and user state. The crate can be embedded inside desktop, mobile, or |
| 7 | +cloud services that need to read and apply AppFlowy collaboration updates. |
7 | 8 |
|
8 | | - |
| 9 | +## Highlights |
| 10 | +- Unified crate that orchestrates the AppFlowy collaboration stack: documents, databases, folders, importer tooling, |
| 11 | + plugins, and user awareness. |
| 12 | +- Strongly typed API layered on yrs, with transaction helpers and schema mappers that keep CRDT updates ergonomic. |
| 13 | +- Pluggable storage and sync via the `plugins` feature (RocksDB for local caches, Supabase sync, plus custom |
| 14 | + extensions). |
| 15 | +- Comprehensive importer and converter support: Markdown/Notion import pipelines, CSV import (optional), and |
| 16 | + markdown/plain text exports. |
| 17 | +- Integration tests that cover document editing, database history, importer pipelines, and plugin behaviour. |
9 | 18 |
|
10 | | -As the project is still a work in progress, it is rapidly evolving to improve its features and functionality. Therefore, |
11 | | -it may still have some bugs and limitations, and its API may change frequently as new features are added and existing |
12 | | -ones are refined. |
| 19 | +## Architecture Overview |
| 20 | +The AppFlowy runtime connects three domain services—`flowy-folder`, `flowy-database`, and `flowy-document`—through the |
| 21 | +`Collab` object supplied by this crate. Core responsibility highlights: |
13 | 22 |
|
14 | | -## collab |
15 | | -The `collab` crate is built on top of the [yrs](https://docs.rs/yrs/latest/yrs/) crate, providing a higher level of |
16 | | -abstraction for the collaborative features of AppFlowy. It exposes cohesive modules: |
| 23 | +1. Client code requests domain operations (e.g. create a view, update a document block) via the domain modules in |
| 24 | + `collab`. |
| 25 | +2. `Collab` records the change as a yrs transaction and fires plugin hooks. |
| 26 | +3. Plugins persist the change (e.g. `RocksdbDiskPlugin`) and optionally sync it to remote services (`SupabaseDBPlugin`, |
| 27 | + custom websocket backends). |
| 28 | +4. `Collab` fans the update out to any other connected clients, keeping local caches and UI layers in sync. |
17 | 29 |
|
18 | | -- Entity definitions and protobuf types under `collab::entity`. |
19 | | -- Database, document, folder, importer, plugin, and user services under `collab::database`, `collab::document`, |
20 | | - `collab::folder`, `collab::importer`, `collab::plugins`, and `collab::user`. |
21 | | -- The plugin subsystem (`collab::plugins`) is guarded by the optional `plugins` Cargo feature; enable it when you |
22 | | - need local/remote persistence helpers. |
| 30 | +The sequence diagrams in `docs/architecture.md` walk through common flows—creating, opening, editing, and synchronising |
| 31 | +documents—and mirror the behaviour of other collab objects such as databases and user awareness. |
23 | 32 |
|
24 | | -With everything consolidated, consumers only need to depend on the `collab` crate to access the full collaborative |
25 | | -feature set. |
| 33 | +## Module Map |
| 34 | +The `collab` crate organises its API into cohesive modules: |
| 35 | + |
| 36 | +- `collab::core` – yrs wrappers, transactions, `Collab` abstractions, plugin registry, snapshot utilities. |
| 37 | +- `collab::document` – block tree, text manipulation, markdown/plain text conversion, rich-text schema helpers. |
| 38 | +- `collab::database` – view schemas, row/value models, history, relation remappers, CSV helpers (with `import_csv` |
| 39 | + feature). |
| 40 | +- `collab::folder` – workspace hierarchy, migrations, folder/view metadata. |
| 41 | +- `collab::importer` – Notion/Markdown import pipelines, async zip tooling, filesystem utilities. |
| 42 | +- `collab::plugins` – trait definitions plus reference implementations (RocksDB disk cache, Supabase sync bridge). |
| 43 | +- `collab::user` – user awareness state (appearance, reminders) and collaboration helpers. |
| 44 | +- `collab::entity` – protobuf-generated entity definitions shared across modules. |
| 45 | +- `collab::tests` (integration) – scenarios that exercise end-to-end flows across the modules. |
| 46 | + |
| 47 | +## Cargo Features |
| 48 | +- `plugins` (default: off) – enables plugin traits and the bundled RocksDB-backed implementations. Required for |
| 49 | + integration tests under `tests/plugins`. |
| 50 | +- `verbose_log` – emits additional tracing when debugging field-level changes. |
| 51 | +- `trace_transact` – instrument yrs transactions for profiling and troubleshooting. |
| 52 | +- `lock_timeout` / `rwlock_reason` – advanced locking diagnostics for long-running async hosts. |
| 53 | +- `import_csv` – unlocks CSV ingestion support in the database importer. |
| 54 | + |
| 55 | +Enable features when building or testing via `cargo build --features plugins`. |
| 56 | + |
| 57 | +## Getting Started |
| 58 | +### Prerequisites |
| 59 | +- Rust `1.85` toolchain (`rustup toolchain install 1.85` if needed). |
| 60 | +- `cargo` and `rustfmt` from the same toolchain. |
| 61 | +- Optional: [`cargo-make`](https://sagiegurari.github.io/cargo-make/) for the coverage task, `grcov` + |
| 62 | + `llvm-tools-preview` for LCOV export. |
| 63 | + |
| 64 | +### Build & Test |
| 65 | +```bash |
| 66 | +# Compile the crate |
| 67 | +cargo build |
| 68 | + |
| 69 | +# Run the full test suite |
| 70 | +cargo test |
| 71 | + |
| 72 | +# Exercise plugin integration tests |
| 73 | +cargo test --features plugins --test plugins |
| 74 | +``` |
| 75 | + |
| 76 | +### Coverage (optional) |
| 77 | +Use the bundled Cargo Make tasks to produce an LCOV report: |
| 78 | + |
| 79 | +```bash |
| 80 | +cargo make check_grcov # once, validates tooling |
| 81 | +cargo make run_coverage # runs instrumented tests |
| 82 | +``` |
| 83 | + |
| 84 | +The coverage report is written to `target/coverage.lcov`. |
| 85 | + |
| 86 | +## Repository Layout |
| 87 | +``` |
| 88 | +collab/ # Collab crate source |
| 89 | + src/ |
| 90 | + core/ # yrs core wrappers and Collab orchestration |
| 91 | + document/ # document model and text tooling |
| 92 | + database/ # database views, rows, and history |
| 93 | + folder/ # workspace/folder hierarchy |
| 94 | + importer/ # Notion/Markdown import logic |
| 95 | + plugins/ # plugin API and reference implementations |
| 96 | + user/ # user awareness collab objects |
| 97 | + tests/ # integration tests (cargo test) |
| 98 | +docs/ # Architecture notes and UML diagrams |
| 99 | +resources/ # Architecture images referenced by the README |
| 100 | +``` |
| 101 | + |
| 102 | +## Further Reading |
| 103 | +- `docs/architecture.md` – deep dive diagrams covering create/open/edit/sync flows and custom collab objects. |
| 104 | +- `collab/tests/` – integration scenarios demonstrating API usage. |
| 105 | +- [`yrs` documentation](https://docs.rs/yrs/latest/yrs/) – underlying CRDT primitives used throughout `collab`. |
| 106 | + |
| 107 | +Contributions are welcome—start by exploring the architecture docs, running the tests, and opening a discussion in the |
| 108 | +AppFlowy community if you plan to introduce new collab objects or plugins. |
0 commit comments