Merge branch 'master' into felix/trajectory-poc

Author: felixpalmer

CONTRIBUTING.md

@@ -10,7 +10,7 @@ PRs and bug reports are welcome, and we are actively looking for new maintainers
 The **master** branch is the active development branch.
 
 Building deck.gl locally from the source requires node.js `>=14`. Further limitations on the Node version may be imposed by [puppeteer](https://github.com/puppeteer/puppeteer#usage) and [headless-gl](https://github.com/stackgl/headless-gl#supported-platforms-and-nodejs-versions).
-We use [yarn](https://yarnpkg.com/en/docs/install) to manage the dependencies of deck.gl.
+We use [yarn](https://yarnpkg.com/en/docs/install) to manage the dependencies of deck.gl, and [volta](https://docs.volta.sh/guide/getting-started) to manage the Node and yarn version.
 
 ```bash
 git checkout master
@@ -20,6 +20,15 @@ yarn test
 
 See [additional instructions](#troubleshooting) for Windows, Linux and Apple M1.
 
+Run the website:
+
+```bash
+yarn build
+cd website
+yarn
+yarn start
+```
+
 Run the layer browser application:
 
 ```bash

bindings/pydeck/examples/geopandas_integration.py

@@ -1,26 +1,42 @@
 import pydeck as pdk
 import geopandas as gpd
 
-world = gpd.read_file(gpd.datasets.get_path("naturalearth_lowres"))
+world = gpd.read_file("https://naciscdn.org/naturalearth/110m/cultural/ne_110m_admin_0_countries.zip")
+# deck.gl is only compatible with WGS84
+world = world.to_crs("EPSG:4326")
+# Convert any multi polygons into individual polygons
+world = world.explode()
 
-centroids = gpd.GeoDataFrame()
-centroids["geometry"] = world.geometry.centroid
-centroids["name"] = world.name
+centroids = gpd.GeoDataFrame(geometry=world.geometry.centroid)
+centroids["name"] = world.NAME
 
 layers = [
+    # Black background of the country polygons
     pdk.Layer(
         "GeoJsonLayer",
         data=world,
         get_fill_color=[0, 0, 0],
     ),
+
+    # # Alternative way using PolygonLayer, should the above not work
+    # pdk.Layer(
+    #     "PolygonLayer",
+    #     data=world,
+    #     get_polygon="geometry.coordinates",
+    #     get_fill_color=[0, 0, 0],
+    # ),
+
+
+    # Overlay country names at their centroids.
     pdk.Layer(
         "TextLayer",
         data=centroids,
+        # Use this to get geometry coordinates out of a raw GeoDataFrame
         get_position="geometry.coordinates",
         get_size=16,
         get_color=[255, 255, 255],
         get_text="name",
     ),
 ]
 
-pdk.Deck(layers, map_provider=None).to_html("geopandas_integration.html", css_background_color="cornflowerblue")
+pdk.Deck(layers, map_provider=None).to_html("geopandas_integration.html", css_background_color="cornflowerblue", open_browser=True)

bindings/pydeck/examples/polygon_layer.py

@@ -57,7 +57,7 @@ def calculate_elevation(val):
 # Add sunlight shadow to the polygons
 sunlight = {
     "@@type": "_SunLight",
-    "timestamp": 1564696800000,  # Date.UTC(2019, 7, 1, 22),
+    "timestamp": 1564696800000,  # datetime.datetime(2019, 7, 1, 22, tzinfo=datetime.timezone.utc).timestamp() * 1000
     "color": [255, 255, 255],
     "intensity": 1.0,
     "_shadow": True,

bindings/pydeck/pydeck/bindings/widget.py

@@ -1,3 +1,4 @@
+import uuid
 from .json_tools import JSONMixin
 
 TYPE_IDENTIFIER = "@@type"
@@ -30,7 +31,7 @@ class Widget(JSONMixin):
 
     def __init__(self, type, id=None, placement=None, view_id=None, **kwargs):
         self.type = type
-        self.id = id
+        self.id = id or str(uuid.uuid4())
         self.placement = placement
         self.view_id = view_id
         self.__dict__.update(kwargs)

bindings/pydeck/tests/bindings/test_widget.py

@@ -4,5 +4,5 @@
 
 
 def test_widget_constructor():
-    EXPECTED = {"@@type": "ZoomWidget", "placement": "top-right"}
-    assert json.loads(Widget(type="ZoomWidget", placement="top-right", view_id=None).to_json()) == EXPECTED
+    EXPECTED = {"@@type": "ZoomWidget", "placement": "top-right", "id": "test-widget"}
+    assert json.loads(Widget(type="ZoomWidget", placement="top-right", view_id=None, id="test-widget").to_json()) == EXPECTED

docs/api-reference/core/view.md

@@ -81,18 +81,37 @@ The `viewState` property is intended to support a number of use cases:
 * Overriding a partial set of view state properties from a selected view state.
 
 
-#### `clear` (boolean | object, optional) {#clear}
+#### `clear` (boolean, optional) {#clear}
 
-Clears the contents (pixels) of the viewport. The value of the `clear` prop is passed as an argument to luma.gl's `clear` function. If `true` clears color and depth buffers. If an object, behaviour is controlled by the following fields:
+Clears the contents (pixels) of the viewport. If `true` clears color, depth, and stencil buffers. Behavior is controlled with the `clearColor`, `clearDepth`, and `clearStencil` properties.
 
-* `color` (boolean or Array) - if not `false`, clears all active color buffers with either the provided color or the currently set clear color.
-* `depth` (boolean)  - if `true`, clears the depth buffer.
-* `stencil` (boolean) - if `true` clears the stencil buffer.
-
-Note that deck.gl always clears the screen before each render, and clearing, while cheap, is not totally free. This means that viewports should only specify the `clear` property if they need additional clearing, e.g. because they are rendering on top of another viewport, or want to have a different background color etc.
+Note that deck.gl always clears the screen before each render, and clearing, while cheap, is not totally free. This means that viewports should only clear the viewport if they need additional clearing, e.g. because they are rendering on top of another viewport, or want to have a different background color etc.
 
 Default `false`.
 
+#### `clearColor` (number[] | false, optional) {#clearcolor}
+
+Specifies the color to clear the viewport with, as an array of four numbers `[r, g, b, a?]`. Each channel should be an integer between 0 and 255. For example, `[255, 0, 0, 255]` for opaque red. If `clearColor` is `false`, the depth buffer will not be cleared. If `clear` is set to `false`, this property will be ignored.
+
+Default `[0, 0, 0, 0]` (transparent).
+
+#### `clearDepth` (number | false, optional) {#cleardepth}
+
+Specifies the depth buffer value to clear the viewport with, as number between `0.0` and `1.0`. If `clearDepth` is `false`, the depth buffer will not be cleared. If `clear` is set to `false`, this property will be ignored.
+
+Default `1.0` (far plane).
+
+#### `clearStencil` (number | false, optional) {#clearstencil}
+
+Specifies the stencil buffer value to clear the viewport with, as number between `0` and `255`. If `clearStencil` is `false`, the depth buffer will not be cleared. If `clear` is set to `false`, this property will be ignored.
+
+Default `0` (clear).
+
+**Examples:**
+
+*   Clearing to a solid color: `new View({clear: true, clearColor: [80, 120, 200, 255]})`
+*   Clearing color and stencil but not depth: `new View({clear: true, clearColor: [50, 50, 50, 255], clearDepth: false})`
+*   No clearing at all: `new View({})` or `new View({clear: false})`
 
 
 ## Methods

docs/api-reference/geo-layers/terrain-layer.md

@@ -6,7 +6,7 @@ import {TerrainLayerDemo} from '@site/src/doc-demos/geo-layers';
 
 The `TerrainLayer` reconstructs mesh surfaces from height map images, e.g. [Mapzen Terrain Tiles](https://github.com/tilezen/joerd/blob/master/docs/formats.md), which encodes elevation into R,G,B values.
 
-When `elevationData` is supplied with a URL template, i.e. a string containing `'{x}'` and `'{y}'`, it loads terrain tiles on demand using a `TileLayer` and renders a mesh for each tile. If `elevationData` is an absolute URL, a single mesh is used, and the `bounds` prop is required to position it into the world space.
+When `elevationData` is supplied with a URL template, i.e. a string containing `'{x}'` and `'{y}'` (or `'{-y}'` for TMS tiles), it loads terrain tiles on demand using a `TileLayer` and renders a mesh for each tile. If `elevationData` is an absolute URL, a single mesh is used, and the `bounds` prop is required to position it into the world space.
 
 
 import Tabs from '@theme/Tabs';

docs/api-reference/layers/geojson-layer.md

@@ -501,7 +501,7 @@ import type {BinaryPointFeature} from '@loaders.gl/schema';
 
 data.points = {
   type: 'Point',
-  positions: {value: Float32Array([x0, y0, x1, y1, x2, y2, ...]), size: 2}, // Use size=2 for xy and size=3 for xyz
+  positions: {value: new Float32Array([x0, y0, x1, y1, x2, y2, ...]), size: 2}, // Use size=2 for xy and size=3 for xyz
   // featureIds
   // globalFeatureIds
   // numericProps
@@ -518,8 +518,8 @@ import type {BinaryLineFeature} from '@loaders.gl/schema';
 
 data.lines = {
   type: 'LineString',
-  positions: {value: Float32Array([x0, y0, x1, y1, x2, y2, ...]), size: 2}, // Use size=2 for xy and size=3 for xyz
-  pathIndices: {value: Uint16Array([0, 5, 7, ...]), size: 1}, // First line contains vertex 0-4, second line contains vertex 5-6, ...
+  positions: {value: new Float32Array([x0, y0, x1, y1, x2, y2, ...]), size: 2}, // Use size=2 for xy and size=3 for xyz
+  pathIndices: {value: new Uint16Array([0, 5, 7, ...]), size: 1}, // First line contains vertex 0-4, second line contains vertex 5-6, ...
   // featureIds
   // globalFeatureIds
   // numericProps
@@ -535,9 +535,9 @@ Polygons are an extension of the idea introduced with lines, but instead of `pat
 import type {BinaryPolygonFeature} from '@loaders.gl/schema';
 
 data.polygons = {
-  positions: {value: Float32Array([x0, y0, x1, y1, x2, y2, ...]), size: 2}, // Use size=2 for xy and size=3 for xyz
-  polygonIndices: {value: Uint16Array([0, 100, ...]), size: 1}, // First polygon contains vertex 0-99
-  primitivePolygonIndices: {value: Uint16Array([0, 60, 80, 100, ...]), size: 1}, // First polygon has 2 holes, made of vertex 60-79 and vertex 80-99
+  positions: {value: new Float32Array([x0, y0, x1, y1, x2, y2, ...]), size: 2}, // Use size=2 for xy and size=3 for xyz
+  polygonIndices: {value: new Uint16Array([0, 100, ...]), size: 1}, // First polygon contains vertex 0-99
+  primitivePolygonIndices: {value: new Uint16Array([0, 60, 80, 100, ...]), size: 1}, // First polygon has 2 holes, made of vertex 60-79 and vertex 80-99
   // featureIds
   // globalFeatureIds
   // numericProps
@@ -569,13 +569,13 @@ data.points = {
   // featureIds
   // globalFeatureIds
   numericProps: {
-    population: {value: Float32Array([value0, value1, ...], size: 1}
+    population: {value: new Float32Array([value0, value1, ...]), size: 1}
   },
   // properties
  } as BinaryPointFeature
 ```
 
-Both `properties` and `numericProps` are specified per-feature.
+Both `properties` and `numericProps` are specified per-vertex.
 
 #### Feature IDs and global feature IDs
 
@@ -629,13 +629,13 @@ const geojson: FeatureCollection = {
 const binary: BinaryPointFeature = {
   shape: 'binary-feature-collection',
   points: {
-    positions: {value: Float32Array([1,23, 4.56]), size: 2},
+    positions: {value: new Float32Array([1,23, 4.56]), size: 2},
     properties: [{name: 'London'}],
     numericProps: {
-      population: {value: Float32Array([10000000], size: 1}
+      population: {value: new Float32Array([10000000]), size: 1}
     },
-    featureIds: {value: Uint16Array([0]), size: 1},
-    globalFeatureIds: {value: Uint16Array([0]), size: 1},
+    featureIds: {value: new Uint16Array([0]), size: 1},
+    globalFeatureIds: {value: new Uint16Array([0]), size: 1},
     fields: [{id: 123}]
   }
 };

docs/developer-guide/custom-layers/writing-shaders.md

@@ -3,12 +3,12 @@
 A shader library facilitates creating shaders that work seamlessly with deck.gl. The `modules` parameter passed to the [Model](https://github.com/visgl/luma.gl/blob/8.0-release/docs/api-reference/engine/model.md) class can dynamically include parts from this library into your own GLSL code:
 
 ```js
-import {picking, project32, gouraudLighting} from '@deck.gl/core';
+import {picking, project32, gouraudMaterial} from '@deck.gl/core';
 
 const model = new Model(gl, {
   vs: '// vertex shader GLSL source'
   fs: '// fragment shader GLSL source',
-  modules: [picking, project32, gouraudLighting] // list of optional module names
+  modules: [picking, project32, gouraudMaterial] // list of optional module names
 });
 ```
 
@@ -34,8 +34,8 @@ The `project` module also has two extensions, [project32](../../api-reference/co
 
 A simple lighting package is provided in deck.gl, supporting a single directional light in addition to ambient light. Turning on lighting requires normals to be provided for each vertex. There are two flavors:
 
-- [gouraudLighting] - for lighting calculated in the vertex shader
-- [phongLighting] - for lighting calculated in the fragment shader
+- [gouraudMaterial] - for lighting calculated in the vertex shader
+- [phongMaterial] - for lighting calculated in the fragment shader
 
 
 #### fp64

docs/developer-guide/webgpu.md

@@ -0,0 +1,54 @@
+# WebGPU
+
+:::caution
+WebGPU support in deck.gl v9 is a work in progress and is not production ready. At this stage, it is aimed at early adopters who wants to try out the new technology, contributors interested in supporting development, or general users that are curious to understand what to expect from future releases.
+:::
+
+deck.gl is gradually adding support for running on WebGPU. WebGPU support will materialize over successive releases, layer by layer and feature by feature.
+
+## Enabling WebGPU
+
+deck.gl needs to be set up to use a luma.gl device that uses the luma.gl `webgpuAdapter`.
+
+One way to do this is to supply `deviceProps` to the Deck constructor:
+
+```ts
+import {webgpuAdapter} from '@luma.gl/webgpu`;
+
+new Deck({
+    deviceProps: {
+        type: 'webgpu',
+        adapters: [webgpuAdapter]
+    }
+});
+```
+
+## Layers
+
+Layers will gradually be WebGPU enabled. Layers that support WebGPU can be tested in the deck.gl website, using the WebGPU tab.
+
+- [`PointCloudLayer`](https://deck.gl/examples/point-cloud-layer)
+- [`ScatterplotLayer`](https://deck.gl/examples/scatterplot-layer)
+
+## Features
+
+Apart from listing which layers have been ported, another aspect is what deck.gl features support WebGPU. 
+
+| Feature               | Status | Comment                                                                                 |
+| --------------------- | ------ | --------------------------------------------------------------------------------------- |
+| Views                 | 🚧      | Various view systems should work via the WGSL port of the `project module`.ß             |
+| Picking               | ❌      | WebGPU pixel readout is asynchronous, requiring an overhaul of deck.gl's picking engine |
+| Base Map overlays     | ❌      | deck.gl needs to adopt pre-multiplied colors to enable transparency                     |
+| Base Map interleaving | ❌      | At the moment no basemaps support WebGPU                                                |
+
+This is currently an incomplete list. Expect more detail in upcoming releases.
+
+## Background
+
+While the WebGPU support in deck.gl may look limited, under the hood, WebGPU enablement has been in progress for several years and a lot of work has been completed. In particular [luma.gl](https://luma.gl), the GPU framework powering deck.gl, has been rewritten and adopted a "WebGPU-first" design.
+
+## Participating
+
+If you want to contribute to deck.gl WebGPU development, or just follow along, we have a dedicated channel in our OpenJS / Open Visualization slack channel.
+
+You can also check various release tracker tasks on GitHub.