Skip to content

Commit 32eed30

Browse files
authored
improved readme (#95)
1 parent 31ad3fa commit 32eed30

File tree

1 file changed

+107
-1
lines changed

1 file changed

+107
-1
lines changed

README.md

Lines changed: 107 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,113 @@
11
# @geoarrow/deck.gl-layers
22

3-
[deck.gl](https://deck.gl) layers for highly efficient rendering with [GeoArrow](https://geoarrow.org) data.
3+
The easiest, most efficient way to render large geospatial datasets in [deck.gl](https://deck.gl), via [GeoArrow](https://geoarrow.org).
44

55
![](assets/hero.jpg)
66

77
<p style="text-align:center">3.2 million points rendered with a <code>GeoArrowScatterplotLayer</code>.</p>
8+
9+
## Features
10+
11+
- **Fast**: copies binary buffers directly from an [Arrow JS](https://www.npmjs.com/package/apache-arrow) [`Table`](https://arrow.apache.org/docs/js/classes/Arrow_dom.Table.html) object to the GPU using [deck.gl's binary data interface](https://deck.gl/docs/developer-guide/performance#supply-attributes-directly).
12+
- **Memory-efficient**: no intermediate data representation and no garbage-collector overhead.
13+
- **Full layer customization**: Use the same layer properties as in the upstream deck.gl layer documentation. Any _accessor_ (layer property prefixed with `get*`) can be passed an Arrow [`Vector`](https://arrow.apache.org/docs/js/classes/Arrow_dom.Vector.html).
14+
- **Input validation**. Validation can be turned off via the `_validate` property on most layer types.
15+
- **Multi-threaded polygon triangulation**. When rendering polygon layers, a process called [polygon triangulation](https://en.wikipedia.org/wiki/Polygon_triangulation) must happen on the CPU before data can be copied to the GPU. Ordinarily, this can block the main thread for several seconds, but the `GeoArrowSolidPolygonLayer` will perform this process off the main thread, on up to 8 web workers.
16+
- **Progressive rendering support**. For streaming-capable data formats like Arrow IPC and Parquet, you can render a GeoArrow layer per chunk as the data loads.
17+
18+
## Examples
19+
20+
Standalone examples exist in the [`examples/`](examples/) directory. Create an issue if you have trouble running them.
21+
22+
More hosted examples on Observable are planned.
23+
24+
## Data Loading
25+
26+
To create deck.gl layers using this library, you need to first get GeoArrow-formatted data into the browser, discussed below.
27+
28+
[OGR/GDAL](https://gdal.org/) is useful for converting among data formats on the backend, and it includes both [GeoArrow](https://gdal.org/drivers/vector/arrow.html#vector-arrow) and [GeoParquet](https://gdal.org/drivers/vector/parquet.html) drivers. Pass `-lco GEOMETRY_ENCODING=GEOARROW` when converting to Arrow or Parquet files in order to store geometries in a GeoArrow-native geometry column.
29+
30+
### Arrow IPC
31+
32+
If you already have Arrow IPC files (also called Feather files) with a GeoArrow geometry column, you can use [`apache-arrow`](https://www.npmjs.com/package/apache-arrow) to load those files.
33+
34+
```ts
35+
import { tableFromIPC } from "apache-arrow";
36+
import { GeoArrowScatterplotLayer } from "@geoarrow/deck.gl-layers";
37+
38+
const resp = await fetch("url/to/file.arrow");
39+
const jsTable = await tableFromIPC(resp);
40+
const deckLayer = new GeoArrowScatterplotLayer({
41+
data: jsTable,
42+
/// Replace with the correct geometry column name
43+
getPosition: jsTable.getChild("geometry")!,
44+
});
45+
```
46+
47+
Note those IPC files must be saved **uncompressed** (at least not internally compressed). As of v14, Arrow JS does not currently support loading IPC files with internal compression.
48+
49+
### Parquet
50+
51+
If you have a Parquet file where the geometry column is stored as _GeoArrow_ encoding (i.e. not as a binary column with WKB-encoded geometries), you can use the stable `parquet-wasm` library to load those files.
52+
53+
```ts
54+
import { readParquet } from "parquet-wasm"
55+
import { tableFromIPC } from "apache-arrow";
56+
import { GeoArrowScatterplotLayer } from "@geoarrow/deck.gl-layers";
57+
58+
const resp = await fetch("url/to/file.parquet");
59+
const arrayBuffer = await resp.arrayBuffer();
60+
const wasmTable = readParquet(new Uint8Array(arrayBuffer));
61+
const jsTable = tableFromIPC(wasmTable.intoIPCStream());
62+
const deckLayer = new GeoArrowScatterplotLayer({
63+
data: jsTable,
64+
/// Replace with the correct geometry column name
65+
getPosition: jsTable.getChild("geometry")!,
66+
});
67+
```
68+
69+
See below for instructions to load GeoParquet 1.0 files, which have WKB-encoded geometries that need to be decoded before they can be used with `@geoarrow/deck.gl-layers`.
70+
71+
### GeoParquet
72+
73+
An initial version of the [`@geoarrow/geoparquet-wasm`](https://www.npmjs.com/package/@geoarrow/geoparquet-wasm) library is published, which reads a GeoParquet file to GeoArrow memory.
74+
75+
```ts
76+
import { readGeoParquet } from "@geoarrow/geoparquet-wasm";
77+
import { tableFromIPC } from "apache-arrow";
78+
import { GeoArrowScatterplotLayer } from "@geoarrow/deck.gl-layers";
79+
80+
const resp = await fetch("url/to/file.parquet");
81+
const arrayBuffer = await resp.arrayBuffer();
82+
const wasmTable = readGeoParquet(new Uint8Array(arrayBuffer));
83+
const jsTable = tableFromIPC(wasmTable.intoTable().intoIPCStream());
84+
const deckLayer = new GeoArrowScatterplotLayer({
85+
data: jsTable,
86+
/// Replace with the correct geometry column name
87+
getPosition: jsTable.getChild("geometry")!,
88+
});
89+
```
90+
91+
If you hit a bug with `@geoarrow/geoparquet-wasm`, please create a reproducible bug report [here](https://github.com/geoarrow/geoarrow-rs/issues/new).
92+
93+
### FlatGeobuf
94+
95+
An initial version of the [`@geoarrow/flatgeobuf-wasm`](https://www.npmjs.com/package/@geoarrow/flatgeobuf-wasm) library is published, which reads a FlatGeobuf file to GeoArrow memory. As of version 0.2.0-beta.1, this library does not yet support remote files, and expects the full FlatGeobuf file to exist in memory.
96+
97+
```ts
98+
import { readFlatGeobuf } from "@geoarrow/flatgeobuf-wasm";
99+
import { tableFromIPC } from "apache-arrow";
100+
import { GeoArrowScatterplotLayer } from "@geoarrow/deck.gl-layers";
101+
102+
const resp = await fetch("url/to/file.fgb");
103+
const arrayBuffer = await resp.arrayBuffer();
104+
const wasmTable = readFlatGeobuf(new Uint8Array(arrayBuffer));
105+
const jsTable = tableFromIPC(wasmTable.intoTable().intoIPCStream());
106+
const deckLayer = new GeoArrowScatterplotLayer({
107+
data: jsTable,
108+
/// Replace with the correct geometry column name
109+
getPosition: jsTable.getChild("geometry")!,
110+
});
111+
```
112+
113+
If you hit a bug with `@geoarrow/flatgeobuf-wasm`, please create a reproducible bug report [here](https://github.com/geoarrow/geoarrow-rs/issues/new).

0 commit comments

Comments
 (0)