Skip to content

Commit 830d5e3

Browse files
mfisher87arjxn-pybatpadpre-commit-ci[bot]kpdavi
authored
Add architecture docs (#576)
* Init architecture docs Co-authored-by: Arjun Verma <[email protected]> Co-authored-by: Sanjay Bhangar <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix JupyterCAD repo link Co-authored-by: Arjun Verma <[email protected]> * Incorporate improvements from GH thread See: <#576 (comment)> Co-authored-by: Sanjay Bhangar <[email protected]> Co-authored-by: Kristin Davis <[email protected]> Co-authored-by: martinRenou <[email protected]> * Lint 🔔 * Add architecture docs to TOC * Specify use of React --------- Co-authored-by: Arjun Verma <[email protected]> Co-authored-by: Sanjay Bhangar <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Kristin Davis <[email protected]> Co-authored-by: martinRenou <[email protected]>
1 parent 3e2f640 commit 830d5e3

File tree

2 files changed

+147
-0
lines changed

2 files changed

+147
-0
lines changed

‎docs/contributor_guide/architecture.md‎

Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,146 @@
1+
# Architecture overview
2+
3+
JupyterGIS is a JupyterLab extension (based on the structure defined by
4+
[jupyterlab/extensions-cookiecutter-ts](https://github.com/jupyterlab/extension-cookiecutter-ts)).
5+
6+
Its architecture is based on QuantStack's
7+
[JupyterCAD](https://github.com/jupytercad/JupyterCAD) architecture.
8+
9+
## JupyterLab
10+
11+
### About Lumino and JupyterLab
12+
13+
JupyterGIS is a JupyterLab extension. It may be useful to read more about the
14+
[extensions developer documentation](https://jupyterlab.readthedocs.io/en/latest/extension/extension_dev.html).
15+
16+
The [Lumino](https://lumino.readthedocs.io/en/latest/api/index.html) library is a
17+
framework used to control the UI - i.e., tracks what changes in the UI and how it should
18+
react to that change.
19+
20+
## JupyterGIS components and structure
21+
22+
JupyterGIS is a monorepo containing TypeScript and Python packages.
23+
24+
### TypeScript packages
25+
26+
TypeScript packages live in the `packages/` directory.
27+
28+
If you change anything about TypeScript packages, you'll need to rebuild with `jlpm run
29+
build`.
30+
31+
### `@jupytergis/base`
32+
33+
This package contains everything that controls the map using
34+
[OpenLayers](https://openlayers.org/doc/), panels, buttons, dialogs; all as
35+
[React](https://react.dev/) components.
36+
It is a UI library, collection of tools - but it does not do anything by itself.
37+
We use this package to make the JupyterLab extension.
38+
39+
- Defines the map view. See `packages/base/src/mainview`.
40+
- Generates the layer gallery. See
41+
`packages/base/rasterlayer_gallery_generator.py`.
42+
- Defines "commands" that appear in various GUI menus and the command pallette
43+
(`CTRL+SHIFT+C`).
44+
See `packages/base/src/commands/`.
45+
- Defines the toolbar and associated commands.
46+
See `packages/base/src/toolbar/widget.tsx`.
47+
- Generates forms from the schema package.
48+
See `packages/base/src/formbuilder/`.
49+
- Contains all logic related to adding layers and reading data.
50+
51+
### `@jupytergis/schema`
52+
53+
Defines our `.jgis` file format - as JSON schemas.
54+
The source of truth for data structures in JupyterGIS.
55+
If you wish to add a new layer _type_, you would need to add it to the schema.
56+
57+
Python classes and Typescript types are automatically generated from the schema at
58+
build-time (i.e. not commited to the repository) using
59+
[`json2ts`](https://github.com/GregorBiswanger/json2ts) for TypeScript,
60+
and
61+
[`datamodel-code-generator`](https://docs.pydantic.dev/latest/integrations/datamodel_code_generator/)
62+
for Python.
63+
64+
- Forms: Generated from e.g. `schema/src/schema/project/layers/vectorlayer.json`
65+
- Project file / shared model: `schema/src/schema/project/jgis.json`
66+
67+
### Python packages
68+
69+
Python packages live in the `python/` directory.
70+
These Python packages may include some TypeScript as well.
71+
72+
- `jupytergis`: A metapackage including `jupytergis_core`, `jupytergis_lab`,
73+
`jupytergis_qgis`, `jupyter-collaboration`, and `jupyterlab`.
74+
- `jupytergis_lite`: A metapackage including `jupytergis_core` and `jupytergis_lab`.
75+
For deployment and testing of JupyterGIS in JupyterLite.
76+
- `jupytergis_core`: Gets the UI to do things - e.g., load / create JupyterGIS files,
77+
and work with them.
78+
Also includes a server endpoint for saving the created `.jgis` files to disk (not used
79+
in JupyterLite).
80+
- `jupytergis_lab`: Contains everything needed for JupyterGIS to work within a notebook,
81+
**the Python API**, the notebook renderer (the part that displays the JupyterGIS
82+
session in the notebook).
83+
**Might be worth considering renaming this folder? Current name doesn't reflect what
84+
it does**.
85+
- `jupytergis_qgis`: Enables importing and exporting QGIS project files.
86+
Requires a server component, and currently is not used in JupyterLite.
87+
88+
### "Model"
89+
90+
Structure is defined in schema `packages/schema/src/schema/project/jgis.json`.
91+
92+
#### Shared model
93+
94+
All collaborators share this and listen for changes to this.
95+
It mediates changes with Conflict-free Replicated Data Types (CRDTs), which is handled
96+
by `yjs`.
97+
It is the "magic sauce" that enables collaboration!
98+
99+
:::tip
100+
You can view the shared model in many contexts by writing
101+
`console.log(model.sharedModel)` in a TypeScript file!
102+
:::
103+
104+
### Commands
105+
106+
Many new features are a matter of defining a new command.
107+
108+
### Forms
109+
110+
JupyterGIS uses automatically generated forms for creating/editing layers and
111+
more.
112+
113+
Those forms are generated from schema definitions, meaning that adding a new
114+
entry in the schema will automatically create user-facing UI components when
115+
editing layers.
116+
117+
An example of this was [adding a new "interpolate" parameter for raster
118+
sources](https://github.com/geojupyter/jupytergis/pull/522/files), the only
119+
required changes were to add the new schema entry, and react on the
120+
"interpolate" value in the OpenLayers viewer.
121+
122+
Many forms are generated from `BaseForm` (the default form implementation), but
123+
some forms use other classes which extend `BaseForm` in order to provide more
124+
advanced controls.
125+
Each of these classes accepts the relevant schema as a property in order to
126+
generate the form on-the-fly. The correct form class is selected in
127+
`formselector.ts`.
128+
129+
### Map view
130+
131+
JupyterGIS uses [OpenLayers](https://openlayers.org/doc/) as a rendering engine.
132+
133+
The action happens in the `@jupytergis/base` package, at
134+
`packages/base/src/mainview/mainView.tsx`.
135+
136+
#### Swappable rendering engine?
137+
138+
The Venn Diagram of the JavaScript map rendering engine ecosystem unfortunately looks
139+
like a bunch of disparate circles with few overlaps.
140+
The burden of understanding this is very high, and we hope to avoid shifting
141+
this burden on to our users.
142+
143+
For example, OpenLayers has excellent support for alternative map projections and
144+
low-level API, but lacks support for visualizing huge vector datasets with the
145+
GPU.
146+
DeckGL can quickly render huge datasets, but lacks projection support.

‎docs/contributor_guide/index.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ To chat with other contributors, please
1010
:maxdepth: 2
1111
1212
development_setup
13+
architecture
1314
how-tos/index
1415
testing
1516
code_quality

0 commit comments

Comments
 (0)