Skip to content

Commit f9ba75e

Browse files
committed
Use CH.Code component for notebook in-out pairs (#752)
1 parent 5f05b3f commit f9ba75e

File tree

2 files changed

+145
-64
lines changed

2 files changed

+145
-64
lines changed

apps/labs/pages/styles.css

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,3 +11,11 @@
1111
.navbar-open {
1212
@apply h-screen overflow-y-hidden;
1313
}
14+
15+
/*
16+
Hide the top-left buttons in the Code Hike block.
17+
See: https://codehike.org/docs/ch-code#panels
18+
*/
19+
.labs-blog-cell-in-out .ch-frame-buttons {
20+
display: none;
21+
}

apps/labs/posts/uarray-intro.md renamed to apps/labs/posts/uarray-intro.mdx

Lines changed: 137 additions & 64 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
---
2-
title: "`uarray`: A Generic Override Framework for Methods"
2+
title: '`uarray`: A Generic Override Framework for Methods'
33
author: hameer-abbasi
44
published: April 30, 2019
55
description: 'The problem is, stated simply: How do we use all of the PyData libraries in tandem, moving seamlessly from one to the other, without actually changing the API, or even the imports?'
66
category: [PyData ecosystem]
77
featuredImage:
8-
src: /posts/hello-world-post/blog_hero_var1.svg
8+
src: /posts/uarray-intro/blog_feature_var1.svg
99
alt: 'An illustration of a brown and a dark brown hand coming towards each other to pass a business card with the logo of Quansight Labs.'
1010
hero:
11-
imageSrc: /posts/hello-world-post/blog_hero_var1.svg
11+
imageSrc: /posts/uarray-intro/blog_hero_var1.svg
1212
imageAlt: 'An illustration of a brown hand holding up a microphone, with some graphical elements highlighting the top of the microphone.'
1313
---
1414

@@ -31,12 +31,12 @@ GPUs and distributed computing emerged. Also, there were old ideas that
3131
couldn't really be used with NumPy's API, such as sparse arrays. To
3232
solve these problems, various libraries emerged:
3333

34-
- Dask, for distributed NumPy
35-
- CuPy, for NumPy on Nvidia-branded GPUs.
36-
- PyData/Sparse, a project started to make sparse arrays conform to
37-
the NumPy API
38-
- Xnd, which extends the type system and the universal function
39-
concept found in NumPy
34+
- Dask, for distributed NumPy
35+
- CuPy, for NumPy on Nvidia-branded GPUs.
36+
- PyData/Sparse, a project started to make sparse arrays conform to
37+
the NumPy API
38+
- Xnd, which extends the type system and the universal function
39+
concept found in NumPy
4040

4141
There were yet other libraries that emerged: PyTorch, which mimics NumPy
4242
to a certain degree; TensorFlow, which defines its own API; and MXNet,
@@ -53,27 +53,21 @@ Oliphant so eloquently puts it, \"re-writing the world\"?
5353
In my mind, the goals are (stated abstractly):
5454

5555
1. Methods that are not tied to a specific implementation.
56-
57-
- For example `np.arange`
58-
56+
- For example `np.arange`
5957
1. Backends that implement these methods.
60-
61-
- NumPy, Dask, PyTorch are all examples of this.
62-
58+
- NumPy, Dask, PyTorch are all examples of this.
6359
1. Coercion of objects to other forms to move between backends.
64-
65-
- This means converting a NumPy array to a Dask array, and vice versa.
60+
- This means converting a NumPy array to a Dask array, and vice versa.
6661

6762
In addition, we wanted to be able to do this for arbitrary objects. So
6863
`dtype`s, `ufunc`s etc. should also be dispatchable and coercible.
6964

7065
## The Solution?
7166

7267
With that said, let's dive into `uarray`. If you're not interested in
73-
the gory details, you can jump down to
74-
`<a href="#how-to-use-it">`{=html}this section`</a>`{=html}.
68+
the gory details, you can jump down to <a href="#how-to-use-it">this section</a>
7569

76-
``` python
70+
```python
7771
import uarray as ua
7872

7973
# Let's ignore this for now
@@ -99,43 +93,67 @@ def myfunc_be2(): # Note that it has exactly the same signature
9993
return "Strawberry"
10094
```
10195

102-
``` python
96+
<CH.Code className="labs-blog-cell-in-out">
97+
98+
```python in
10399
with ua.set_backend(be1):
104100
print(myfunc())
105101
```
106102

107-
Potato
103+
---
104+
105+
```txt out
106+
Potato
107+
```
108+
109+
</CH.Code>
108110

109-
``` python
111+
<CH.Code className="labs-blog-cell-in-out">
112+
113+
```python in
110114
with ua.set_backend(be2):
111115
print(myfunc())
112116
```
113117

114-
Strawberry
118+
---
119+
120+
```text out
121+
Strawberry
122+
```
123+
124+
</CH.Code>
115125

116126
As we can clearly see: We have already provided a way to do (1) and (2)
117127
above. But then we run across the problem: How do we decide between
118128
these backends? How do we move between them? Let's go ahead and
119129
register both of these backends for permanent use. And see what happens
120130
when we want to implement both of their methods!
121131

122-
``` python
132+
```python
123133
ua.register_backend(be1)
124134
ua.register_backend(be2)
125135
```
126136

127-
``` python
137+
<CH.Code className="labs-blog-cell-in-out">
138+
139+
```python in
128140
print(myfunc())
129141
```
130142

131-
Potato
143+
---
144+
145+
```text out
146+
Potato
147+
```
148+
149+
</CH.Code>
132150

133151
As we see, we get only the first backend's answer. In general, it's
134152
indeterminate what backend will be selected. But, this is a special
135153
case: We're not passing arguments in! What if we change one of these to
136154
return `NotImplemented`?
137155

138-
``` python
156+
```python
139157
# We redefine the multimethod so it's new again
140158
@ua.create_multimethod(myfunc_rd)
141159
def myfunc():
@@ -158,38 +176,54 @@ ua.register_backend(be1)
158176
ua.register_backend(be2)
159177
```
160178

161-
``` python
179+
<CH.Code className="labs-blog-cell-in-out">
180+
181+
```python in
162182
with ua.set_backend(be1):
163183
print(myfunc())
164184
```
165185

166-
Strawberry
186+
---
187+
188+
```txt out
189+
Strawberry
190+
```
191+
192+
</CH.Code>
167193

168194
Wait\... What? Didn't we just set the first `Backend`? Ahh, but, you
169-
see\... It's signalling that it has *no* implementation for `myfunc`.
195+
see\... It's signalling that it has _no_ implementation for `myfunc`.
170196
The same would happen if you simply didn't register one. To force a
171197
`Backend`, we must use `only=True` or `coerce=True`, the difference will
172198
be explained in just a moment.
173199

174-
``` python
200+
<CH.Code className="labs-blog-cell-in-out">
201+
202+
```python in
175203
with ua.set_backend(be1, only=True):
176204
print(myfunc())
177205
```
178206

179-
---------------------------------------------------------------------------
180-
BackendNotImplementedError Traceback (most recent call last)
181-
<ipython-input-8-ec856cf7c88b> in <module>
182-
1 with ua.set_backend(be1, only=True):
183-
----> 2 print(myfunc())
207+
---
184208

185-
~/Quansight/uarray/uarray/backend.py in __call__(self, *args, **kwargs)
186-
108
187-
109 if result is NotImplemented:
188-
--> 110 raise BackendNotImplementedError('No selected backends had an implementation for this method.')
189-
111
190-
112 return result
209+
```txt out
210+
---------------------------------------------------------------------------
211+
BackendNotImplementedError Traceback (most recent call last)
212+
<ipython-input-8-ec856cf7c88b> in <module>
213+
1 with ua.set_backend(be1, only=True):
214+
----> 2 print(myfunc())
215+
216+
~/Quansight/uarray/uarray/backend.py in __call__(self, *args, **kwargs)
217+
108
218+
109 if result is NotImplemented:
219+
--> 110 raise BackendNotImplementedError('No selected backends had an implementation for this method.')
220+
111
221+
112 return result
222+
223+
BackendNotImplementedError: No selected backends had an implementation for this method.
224+
```
191225

192-
BackendNotImplementedError: No selected backends had an implementation for this method.
226+
</CH.Code>
193227

194228
Now we are told that no backends had an implementation for this function
195229
(which is nice, good error messages are nice!)
@@ -199,7 +233,7 @@ Now we are told that no backends had an implementation for this function
199233
Let's say we had two `Backend`s. Let's choose the completely useless
200234
example of one storing a number as an `int` and one as a `float`.
201235

202-
``` python
236+
```python
203237
class Number(ua.DispatchableInstance):
204238
pass
205239

@@ -222,56 +256,82 @@ Number.register_convertor(be2, lambda x: str(x))
222256
Let's also define a \"catch-all\" method. This catches all
223257
implementations of methods not already registered.
224258

225-
``` python
259+
```python
226260
# This can be arbitrarily complex
227261
def gen_impl1(method, args, kwargs, dispatchable_args):
228262
if not all(isinstance(a, Number) and isinstance(a.value, int) for a in dispatchable_args):
229263
return NotImplemented
230-
264+
231265
return args[0]
232266

233267
# This can be arbitrarily complex
234268
def gen_impl2(method, args, kwargs, dispatchable_args):
235269
if not all(isinstance(a, Number) and isinstance(a.value, str) for a in dispatchable_args):
236270
return NotImplemented
237-
271+
238272
return args[0]
239273

240274
be1.register_implementation(None, gen_impl1)
241275
be2.register_implementation(None, gen_impl2)
242276
```
243277

244-
``` python
278+
<CH.Code className="labs-blog-cell-in-out">
279+
280+
```python in
245281
myfunc('1') # This calls the second implementation
246282
```
247283

248-
'1'
284+
---
285+
286+
```txt out
287+
'1'
288+
```
289+
290+
</CH.Code>
291+
292+
<CH.Code className="labs-blog-cell-in-out">
249293

250-
``` python
294+
```python in
251295
myfunc(1) # This calls the first implementation
252296
```
253297

254-
1
298+
---
299+
300+
```txt out
301+
1
302+
```
303+
304+
</CH.Code>
255305

256-
``` python
306+
<CH.Code className="labs-blog-cell-in-out">
307+
308+
```python in
257309
myfunc(1.0) # This fails
258310
```
259311

312+
---
313+
314+
```txt out
260315
---------------------------------------------------------------------------
261316
BackendNotImplementedError Traceback (most recent call last)
262317
<ipython-input-13-8431c1275db5> in <module>
263318
----> 1 myfunc(1.0) # This fails
264319
265320
~/Quansight/uarray/uarray/backend.py in __call__(self, *args, **kwargs)
266-
108
321+
108
267322
109 if result is NotImplemented:
268323
--> 110 raise BackendNotImplementedError('No selected backends had an implementation for this method.')
269-
111
324+
111
270325
112 return result
271326
272327
BackendNotImplementedError: No selected backends had an implementation for this method.
328+
```
273329

274-
``` python
330+
</CH.Code>
331+
332+
<CH.Code className="labs-blog-cell-in-out">
333+
334+
```python in
275335
# But works if we do this:
276336

277337
with ua.set_backend(be1, coerce=True):
@@ -281,8 +341,14 @@ with ua.set_backend(be2, coerce=True):
281341
print(type(myfunc(1.0)))
282342
```
283343

284-
<class 'int'>
285-
<class 'str'>
344+
---
345+
346+
```txt out
347+
<class 'int'>
348+
<class 'str'>
349+
```
350+
351+
</CH.Code>
286352

287353
This may seem like too much work, but remember that it's broken down
288354
into a lot of small steps:
@@ -305,15 +371,23 @@ right values from it.
305371
`unumpy` is a set of NumPy-related multimethods built on top of
306372
`uarray`. You can use them as follows:
307373

308-
``` python
374+
<CH.Code className="labs-blog-cell-in-out">
375+
376+
```python in
309377
import unumpy as np # Note the changed import statement
310378
from unumpy.xnd_backend import XndBackend
311379

312380
with ua.set_backend(XndBackend):
313381
print(type(np.arange(0, 100, 1)))
314382
```
315383

316-
<class 'xnd.array'>
384+
---
385+
386+
```txt out
387+
<class 'xnd.array'>
388+
```
389+
390+
</CH.Code>
317391

318392
And, as you can see, we get back an Xnd array when using a NumPy-like
319393
API. Currently, there are three back-ends: NumPy, Xnd and PyTorch. The
@@ -324,8 +398,7 @@ We are also working on supporting more of the NumPy API, and dispatching
324398
over dtypes.
325399

326400
Feel free to browse the source and open issues at:
327-
<https://github.com/Quansight-Labs/uarray> or shoot me an email at
328-
`<a href="mailto:[email protected]">`{=html}[email protected]`</a>`{=html}
329-
if you want to contact me directly. You can also find the full
330-
documentation at <https://uarray.readthedocs.io/en/latest/>.
401+
https://github.com/Quansight-Labs/uarray or shoot me an email at
331402

403+
404+
if you want to contact me directly. You can also find the full documentation at https://uarray.readthedocs.io/en/latest/.

0 commit comments

Comments
 (0)