Skip to content

Commit 3526b3a

Browse files
authored
Jupyter canvas (#178)
* add jupyter widget * rename a private variable * docs * more control over canvas texture format * fix * add ref to jupyter_rfb examples
1 parent 3de7a06 commit 3526b3a

File tree

6 files changed

+168
-30
lines changed

6 files changed

+168
-30
lines changed

README.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,9 @@ from wgpu.gui.glfw import WgpuCanvas
8080
# Import PySide2, PyQt5, PySide or PyQt4 before running the line below.
8181
# The code will detect and use the library that is imported.
8282
from wgpu.gui.qt import WgpuCanvas
83+
84+
# You can also show wgpu visualizations in Jupyter
85+
from wgpu.gui.jupyter import WgpuCanvas
8386
```
8487

8588
Some functions in the original `wgpu-native` API are async. In the Python API,

docs/reference_gui.rst

Lines changed: 40 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ screen is also possible, but we need a *canvas* for that. Since the Python
66
ecosystem provides many different GUI toolkits, we need an interface.
77

88
For convenience, the wgpu library has builtin support for a few GUI
9-
toolkits. At the moment these include GLFW and Qt.
9+
toolkits. At the moment these include GLFW, Qt, and wx (experimental).
1010

1111

1212
The canvas interface
@@ -15,34 +15,23 @@ The canvas interface
1515
To render to a window, an object is needed that implements the few
1616
functions on the canvas interface, and provide that object to
1717
:func:`request_adapter() <wgpu.request_adapter>`.
18-
This interface makes it possible to hook wgpu-py to any GUI that supports GPU rendering.
18+
This is the minimal interface required to hook wgpu-py to any GUI that supports GPU rendering.
1919

2020
.. autoclass:: wgpu.gui.WgpuCanvasInterface
2121
:members:
2222

2323

24-
The WgpuCanvas classes
25-
----------------------
24+
The WgpuCanvas base class
25+
-------------------------
2626

27-
For each GUI toolkit that wgpu-py has builtin support for, there is a
28-
``WgpuCanvas`` class, which all derive from the following class. This thus
29-
provides a single (simple) API to work with windows.
27+
For each supported GUI toolkit there are specific
28+
``WgpuCanvas`` classes, which are detailed in the following sections.
29+
These all derive from the same base class, which defines the common API.
3030

3131
.. autoclass:: wgpu.gui.WgpuCanvasBase
3232
:members:
3333

3434

35-
Offscreen canvases
36-
------------------
37-
38-
A base class is provided to implement off-screen canvases. Note that you can
39-
render to a texture without using any canvas object, but in some cases it's
40-
convenient to do so with a canvas-like API, which is what this class provides.
41-
42-
.. autoclass:: wgpu.gui.WgpuOffscreenCanvas
43-
:members:
44-
45-
4635
Support for Qt
4736
--------------
4837

@@ -102,3 +91,36 @@ Glfw is a lightweight windowing toolkit. Install it with ``pip install glfw``.
10291
10392
Also see the `GLFW triangle example <https://github.com/pygfx/wgpu-py/blob/main/examples/triangle_glfw.py>`_
10493
and the `async GLFW example <https://github.com/pygfx/wgpu-py/blob/main/examples/triangle_glfw_asyncio.py>`_.
94+
95+
96+
Offscreen canvases
97+
------------------
98+
99+
A base class is provided to implement off-screen canvases. Note that you can
100+
render to a texture without using any canvas object, but in some cases it's
101+
convenient to do so with a canvas-like API.
102+
103+
.. autoclass:: wgpu.gui.WgpuOffscreenCanvas
104+
:members:
105+
106+
107+
Support for Jupyter lab and notebook
108+
------------------------------------
109+
110+
WGPU can be used in Jupyter lab and the Jupyter notebook. This canvas
111+
is based on `jupyter_rfb <https://github.com/vispy/jupyter_rfb>`_ an ipywidget
112+
subclass implementing a remote frame-buffer. There are also some `wgpu examples <https://jupyter-rfb.readthedocs.io/en/latest/examples/>`_.
113+
114+
To implement interaction, create a subclass and overload the ``handle_event()``
115+
method (and call ``super().handle_event(event)``).
116+
117+
118+
.. code-block:: py
119+
120+
from wgpu.gui.jupyter import WgpuCanvas
121+
122+
canvas = WgpuCanvas()
123+
124+
# ... wgpu code
125+
126+
canvas # Use as cell output

examples/triangle.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@
3030
3131
[[stage(vertex)]]
3232
fn vs_main(in: VertexInput) -> VertexOutput {
33-
var positions: array<vec2<f32>, 3> = array<vec2<f32>, 3>(vec2<f32>(0.0, -0.5), vec2<f32>(0.5, 0.5), vec2<f32>(-0.5, 0.7));
33+
var positions = array<vec2<f32>, 3>(vec2<f32>(0.0, -0.5), vec2<f32>(0.5, 0.5), vec2<f32>(-0.5, 0.7));
3434
let index = i32(in.vertex_index);
3535
let p: vec2<f32> = positions[index];
3636

wgpu/gui/base.py

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ class WgpuCanvasInterface:
1616
def __init__(self, *args, **kwargs):
1717
# The args/kwargs are there because we may be mixed with e.g. a Qt widget
1818
super().__init__(*args, **kwargs)
19-
self._present_context = None
19+
self._canvas_context = None
2020

2121
def get_window_id(self):
2222
"""Get the native window id. This is used to obtain a surface id,
@@ -59,13 +59,13 @@ def get_context(self, kind="gpupresent"):
5959
# Note that this function is analog to HtmlCanvas.get_context(), except
6060
# here the only valid arg is 'gpupresent', which is also made the default.
6161
assert kind == "gpupresent"
62-
if self._present_context is None:
62+
if self._canvas_context is None:
6363
# Get the active wgpu backend module
6464
backend_module = sys.modules["wgpu"].GPU.__module__
6565
# Instantiate the context
6666
PC = sys.modules[backend_module].GPUCanvasContext # noqa: N806
67-
self._present_context = PC(self)
68-
return self._present_context
67+
self._canvas_context = PC(self)
68+
return self._canvas_context
6969

7070

7171
class WgpuCanvasBase(WgpuCanvasInterface):
@@ -102,13 +102,14 @@ def _draw_frame_and_present(self):
102102
"""
103103
# Perform the user-defined drawing code. When this errors,
104104
# we should report the error and then continue, otherwise we crash.
105+
# Returns the result of the context's present() call or None.
105106
try:
106107
self.draw_frame()
107108
except Exception as err:
108109
self._log_exception("Draw error", err)
109110
try:
110-
if self._present_context:
111-
self._present_context.present()
111+
if self._canvas_context:
112+
return self._canvas_context.present()
112113
except Exception as err:
113114
self._log_exception("Present error", err)
114115

wgpu/gui/jupyter.py

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
"""
2+
Support for rendering in a Jupyter widget. Provides a widget subclass that
3+
can be used as cell output, or embedded in a ipywidgets gui.
4+
"""
5+
6+
from .offscreen import WgpuOffscreenCanvas
7+
8+
import numpy as np
9+
from jupyter_rfb import RemoteFrameBuffer
10+
11+
12+
class JupyterWgpuCanvas(WgpuOffscreenCanvas, RemoteFrameBuffer):
13+
"""An ipywidgets widget providing a wgpu canvas. Needs the jupyter_rfb library."""
14+
15+
def __init__(self):
16+
super().__init__()
17+
self._pixel_ratio = 1
18+
self._logical_size = 0, 0
19+
self._is_closed = False
20+
21+
# Implementation needed for RemoteFrameBuffer
22+
23+
def handle_event(self, event):
24+
event_type = event.get("event_type", "")
25+
if event_type == "close":
26+
self._is_closed = True
27+
elif event_type == "resize":
28+
self._pixel_ratio = event["pixel_ratio"]
29+
self._logical_size = event["width"], event["height"]
30+
31+
def get_frame(self):
32+
# The _draw_frame_and_present() does the drawing and then calls
33+
# present_context.present(), which calls our present() method.
34+
# The resuls is either a numpy array or None, and this matches
35+
# with what this method is expected to return.
36+
return self._draw_frame_and_present()
37+
38+
# Implementation needed for WgpuCanvasBase
39+
40+
def get_pixel_ratio(self):
41+
return self._pixel_ratio
42+
43+
def get_logical_size(self):
44+
return self._logical_size
45+
46+
def get_physical_size(self):
47+
return int(self._logical_size[0] * self._pixel_ratio), int(
48+
self._logical_size[1] * self._pixel_ratio
49+
)
50+
51+
def set_logical_size(self, width, height):
52+
self.css_width = f"{width}px"
53+
self.css_height = f"{height}px"
54+
55+
def close(self):
56+
RemoteFrameBuffer.close(self)
57+
58+
def is_closed(self):
59+
return self._is_closed
60+
61+
def _request_draw(self):
62+
RemoteFrameBuffer.request_draw(self)
63+
64+
# Implementation needed for WgpuOffscreenCanvas
65+
66+
def present(self, texture_view):
67+
# This gets called at the end of a draw pass via GPUCanvasContextOffline
68+
device = texture_view._device
69+
size = texture_view.size
70+
bytes_per_pixel = 4
71+
data = device.queue.read_texture(
72+
{
73+
"texture": texture_view.texture,
74+
"mip_level": 0,
75+
"origin": (0, 0, 0),
76+
},
77+
{
78+
"offset": 0,
79+
"bytes_per_row": bytes_per_pixel * size[0],
80+
"rows_per_image": size[1],
81+
},
82+
size,
83+
)
84+
return np.frombuffer(data, np.uint8).reshape(size[1], size[0], 4)
85+
86+
def get_preferred_format(self):
87+
# Use a format that maps well to PNG: rgba8norm.
88+
# Use srgb for perseptive color mapping. You probably want to
89+
# apply this before displaying on screen, but you do not want
90+
# to duplicate it. When a PNG is shown on screen in the browser
91+
# it's shown as-is (at least it was when I just tried).
92+
return "rgba8unorm-srgb"
93+
94+
95+
# Make available under a name that is the same for all gui backends
96+
WgpuCanvas = JupyterWgpuCanvas

wgpu/gui/offscreen.py

Lines changed: 21 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -14,17 +14,29 @@ def get_window_id(self):
1414

1515
def get_context(self, kind="gpupresent"):
1616
"""Get the GPUCanvasContext object to obtain a texture to render to."""
17+
# Normally this creates a GPUCanvasContext object provided by
18+
# the backend (e.g. rs), but here we use our own context.
1719
assert kind == "gpupresent"
18-
if self._present_context is None:
19-
self._present_context = GPUCanvasContextOffline(self)
20-
return self._present_context
20+
if self._canvas_context is None:
21+
self._canvas_context = GPUCanvasContextOffline(self)
22+
return self._canvas_context
2123

2224
def present(self, texture_view):
2325
"""Method that gets called at the end of each draw event. Subclasses
2426
should provide the approproate implementation.
2527
"""
2628
pass
2729

30+
def get_preferred_format(self):
31+
"""Get the preferred format for this canvas. This method can
32+
be overloaded to control the used texture format. The default
33+
is "rgba8unorm" (not including srgb colormapping).
34+
"""
35+
# Use rgba because that order is more common for processing and storage.
36+
# Use 8unorm because 8bit is enough and common in most cases.
37+
# We DO NOT use srgb colormapping here; we return the "raw" output.
38+
return "rgba8unorm"
39+
2840

2941
class GPUCanvasContextOffline(base.GPUCanvasContext):
3042
"""Helper class for canvases that render to a texture."""
@@ -41,7 +53,11 @@ def unconfigure(self):
4153
self._texture = None
4254

4355
def get_preferred_format(self, adapter):
44-
return "rgba8unorm"
56+
canvas = self._get_canvas()
57+
if canvas:
58+
return canvas.get_preferred_format()
59+
else:
60+
return "rgba8unorm"
4561

4662
def get_current_texture(self):
4763
self._create_new_texture_if_needed()
@@ -51,7 +67,7 @@ def get_current_texture(self):
5167
def present(self):
5268
if self._texture_view is not None:
5369
canvas = self._get_canvas()
54-
canvas.present(self._texture_view)
70+
return canvas.present(self._texture_view)
5571

5672
def _create_new_texture_if_needed(self):
5773
canvas = self._get_canvas()

0 commit comments

Comments
 (0)