diff --git a/DEVELOP.md b/DEVELOP.md new file mode 100644 index 000000000..4e000ad09 --- /dev/null +++ b/DEVELOP.md @@ -0,0 +1 @@ +# Contributing Documentation diff --git a/dev-docs/README.md b/dev-docs/README.md new file mode 100644 index 000000000..43d67fba0 --- /dev/null +++ b/dev-docs/README.md @@ -0,0 +1,6 @@ +# Developer Documentation + +This folder contains documentation on the architecture of `geoarrow-rs` and how to contribute. + +- [Install and set up Rust](./rust-setup.md) +- [Project architecture](./architecture/README.md) diff --git a/dev-docs/architecture/README.md b/dev-docs/architecture/README.md new file mode 100644 index 000000000..fedf84542 --- /dev/null +++ b/dev-docs/architecture/README.md @@ -0,0 +1,4 @@ +# Project Architecture + +- [Rust core](./rust-core.md) +- [JavaScript bindings](./js-bindings.md) diff --git a/dev-docs/architecture/geoarrow-spec.md b/dev-docs/architecture/geoarrow-spec.md new file mode 100644 index 000000000..792d60054 --- /dev/null +++ b/dev-docs/architecture/geoarrow-spec.md @@ -0,0 +1 @@ +# diff --git a/dev-docs/architecture/js-bindings.md b/dev-docs/architecture/js-bindings.md new file mode 100644 index 000000000..2749f6fea --- /dev/null +++ b/dev-docs/architecture/js-bindings.md @@ -0,0 +1,49 @@ +# JavaScript Bindings Architecture + +The goal of the bindings is to be as slim as possible. As much as possible should be included in the core Rust code. + +The JS bindings use [wasm-bindgen](https://rustwasm.github.io/docs/wasm-bindgen/) to generate a WebAssembly binary plus JavaScript functions (plus TypeScript types!) to interact with the Wasm. + +## Arrays + +... + +One thing to keep an eye on is that the bindings have _another set_ of struct names that coincide with the core rust binding! So for example, there's a `LineStringArray` in the JS bindings. It needs to have that name so that JS gets that name, but this is a _different_ struct than the Rust core. So you can't use them interchangeably; you need to convert from one to the other (this conversion is `O(1)`). + +## Scalars + +Scalars are not yet implemented for JS. We need to figure out the right way to abstract over a scalar + +## Algorithms + +Algorithms are defined in individual modules which implement algorithms onto the existing wasm-bindgen structs. + +Ideally the algorithm binding should be extremely minimal. For example, implementing `euclidean_length` in JS happens in [`euclidean_length.rs`](../../js/src/algorithm/geo/euclidean_length.rs). The _entire_ binding for this function is + +```rs +#[wasm_bindgen] +impl LineStringArray { + /// Calculation of the length of a Line + #[wasm_bindgen(js_name = euclideanLength)] + pub fn euclidean_length(&self) -> FloatArray { + use geoarrow::algorithm::geo::EuclideanLength; + FloatArray(EuclideanLength::euclidean_length(&self.0)) + } +} +``` + +Since the implementation is exactly the same for multiple geometry types, we use a macro to deduplicate the binding for multiple geometry array types. + +In this example, you can see you need to apply `#[wasm_bindgen]` on the `impl`, as well as another `#[wasm_bindgen]` on the function itself. We rename the function for JS so that the Rust side can have idiomatic snake case naming, while the JS side has idiomatic camel case naming. + +## Arrow FFI + +Arrow defines a single memory specification for every implementation. This means that the way Arrow memory is laid out in WebAssembly's memory space is the same as in JavaScript's own memory. This means we can use the JS Arrow implementation to interpret Arrow memory from inside Wasm memory. + +I [wrote more on this in a blog post](https://observablehq.com/@kylebarron/zero-copy-apache-arrow-with-webassembly), and have a [JS library here](https://github.com/kylebarron/arrow-js-ffi) that implements the reading across the Wasm boundary. + +All that's needed on the Rust side is to create the structs that fulfill the [C Data Interface](https://arrow.apache.org/docs/format/CDataInterface.html). The code in the [`ffi` mod](../../js/src/ffi/) contains the `FFIArrowArray`, which stores pointers to both the [`ArrowSchema`](https://arrow.apache.org/docs/format/CDataInterface.html#the-arrowschema-structure) struct and the [`ArrowArray`](https://arrow.apache.org/docs/format/CDataInterface.html#the-arrowarray-structure) struct. Those can be read from JS using `arrow-js-ffi`. + +## API Documentation + +Anything documented with `///` in Rust gets converted to a [JSDoc comment](https://github.com/jsdoc/jsdoc) within the exported TypeScript `.d.ts` type file. We then use [TypeDoc](https://github.com/TypeStrong/typedoc) to generate an API documentation website from these types and docstrings. diff --git a/dev-docs/architecture/rust-core.md b/dev-docs/architecture/rust-core.md new file mode 100644 index 000000000..5defc7d42 --- /dev/null +++ b/dev-docs/architecture/rust-core.md @@ -0,0 +1,19 @@ +# Rust Core Architecture + +## Layout + +- `array`. The `array` module contains all the + +``` +. +├── algorithm +├── array +├── error.rs +├── geo_traits +├── io +├── lib.rs +├── scalar +├── test +├── trait_.rs +└── util.rs +``` diff --git a/dev-docs/rust-setup.md b/dev-docs/rust-setup.md new file mode 100644 index 000000000..59d36e879 --- /dev/null +++ b/dev-docs/rust-setup.md @@ -0,0 +1,53 @@ +# Rust Setup + +## Install Rust + +Follow the [official instructions](https://www.rust-lang.org/tools/install) to install Rust on your system. + +## Editor environment + +I use and recommend VSCode with the stellar [Rust Analyzer](https://marketplace.visualstudio.com/items?itemName=rust-lang.rust-analyzer) extension. + +Note that Rust Analyzer works with the _currently active Cargo workspace_. This means that if you open this repository from the root, Rust Analyzer completions will work for the pure-Rust library, but **not for the JavaScript bindings** because the JS bindings are a _separate Cargo project_. You need to open VSCode from the `/js` folder in order for the completions to work for the JS bindings. + +## System dependencies + +One of the examples uses GDAL, and so you might need to have GDAL (3.6+) installed on your system, even if you're not running that example (I'm not sure). + +No other system dependencies are required to my knowledge. + +## Run tests + +We use the default Cargo test runner, so just run: + +```bash +cargo test --all-features +``` + +## Run linter + +We use the default Cargo linter, so just run: + +```bash +cargo clippy --all-features +``` + +## Format code + +Cargo includes a default code formatter. If the code hasn't been formatted, it won't pass CI. + +```bash +cargo fmt +``` + +## View crate documentation + +Any object documented with `///` will be automatically documented. + +To see the current crate documentation locally, run + +```bash +cargo doc --open +``` + +See the sections in the [Rust Book](https://doc.rust-lang.org/book/ch14-02-publishing-to-crates-io.html#making-useful-documentation-comments) and the [Rustdoc guide](https://doc.rust-lang.org/rustdoc/index.html) for all the syntax supported in code comments. diff --git a/js/src/ffi/to_ffi.rs b/js/src/ffi/to_ffi.rs index 5a6aca41f..bc1783988 100644 --- a/js/src/ffi/to_ffi.rs +++ b/js/src/ffi/to_ffi.rs @@ -4,6 +4,7 @@ use arrow2::datatypes::Field; use geoarrow::GeometryArrayTrait; use wasm_bindgen::prelude::*; +/// Implement exporting to an FFI struct for GeoArrow arrays macro_rules! impl_to_ffi { ($struct_name:ident) => { #[wasm_bindgen] @@ -26,6 +27,7 @@ impl_to_ffi!(MultiLineStringArray); impl_to_ffi!(MultiPolygonArray); impl_to_ffi!(GeometryArray); +/// Implement exporting to an FFI struct for native Arrow2 arrays macro_rules! impl_to_ffi_arrow2 { ($struct_name:ident) => { #[wasm_bindgen]