diff --git a/apps/labs/pages/styles.css b/apps/labs/pages/styles.css index 78f032919..153aa16a3 100644 --- a/apps/labs/pages/styles.css +++ b/apps/labs/pages/styles.css @@ -11,3 +11,11 @@ .navbar-open { @apply h-screen overflow-y-hidden; } + +/* + Hide the top-left buttons in the Code Hike block. + See: https://codehike.org/docs/ch-code#panels +*/ +.labs-blog-cell-in-out .ch-frame-buttons { + display: none; +} diff --git a/apps/labs/posts/uarray-intro.md b/apps/labs/posts/uarray-intro.mdx similarity index 76% rename from apps/labs/posts/uarray-intro.md rename to apps/labs/posts/uarray-intro.mdx index c2de1d141..f2c59c153 100644 --- a/apps/labs/posts/uarray-intro.md +++ b/apps/labs/posts/uarray-intro.mdx @@ -1,14 +1,14 @@ --- -title: "`uarray`: A Generic Override Framework for Methods" +title: '`uarray`: A Generic Override Framework for Methods' author: hameer-abbasi published: April 30, 2019 description: 'The problem is, stated simply: How do we use all of the PyData libraries in tandem, moving seamlessly from one to the other, without actually changing the API, or even the imports?' category: [PyData ecosystem] featuredImage: - src: /posts/hello-world-post/blog_hero_var1.svg + src: /posts/uarray-intro/blog_feature_var1.svg alt: 'An illustration of a brown and a dark brown hand coming towards each other to pass a business card with the logo of Quansight Labs.' hero: - imageSrc: /posts/hello-world-post/blog_hero_var1.svg + imageSrc: /posts/uarray-intro/blog_hero_var1.svg imageAlt: 'An illustration of a brown hand holding up a microphone, with some graphical elements highlighting the top of the microphone.' --- @@ -31,12 +31,12 @@ GPUs and distributed computing emerged. Also, there were old ideas that couldn't really be used with NumPy's API, such as sparse arrays. To solve these problems, various libraries emerged: -- Dask, for distributed NumPy -- CuPy, for NumPy on Nvidia-branded GPUs. -- PyData/Sparse, a project started to make sparse arrays conform to - the NumPy API -- Xnd, which extends the type system and the universal function - concept found in NumPy +- Dask, for distributed NumPy +- CuPy, for NumPy on Nvidia-branded GPUs. +- PyData/Sparse, a project started to make sparse arrays conform to + the NumPy API +- Xnd, which extends the type system and the universal function + concept found in NumPy There were yet other libraries that emerged: PyTorch, which mimics NumPy to a certain degree; TensorFlow, which defines its own API; and MXNet, @@ -53,16 +53,11 @@ Oliphant so eloquently puts it, \"re-writing the world\"? In my mind, the goals are (stated abstractly): 1. Methods that are not tied to a specific implementation. - -- For example `np.arange` - + - For example `np.arange` 1. Backends that implement these methods. - -- NumPy, Dask, PyTorch are all examples of this. - + - NumPy, Dask, PyTorch are all examples of this. 1. Coercion of objects to other forms to move between backends. - -- This means converting a NumPy array to a Dask array, and vice versa. + - This means converting a NumPy array to a Dask array, and vice versa. In addition, we wanted to be able to do this for arbitrary objects. So `dtype`s, `ufunc`s etc. should also be dispatchable and coercible. @@ -70,10 +65,9 @@ In addition, we wanted to be able to do this for arbitrary objects. So ## The Solution? With that said, let's dive into `uarray`. If you're not interested in -the gory details, you can jump down to -``{=html}this section``{=html}. +the gory details, you can jump down to this section -``` python +```python import uarray as ua # Let's ignore this for now @@ -99,19 +93,35 @@ def myfunc_be2(): # Note that it has exactly the same signature return "Strawberry" ``` -``` python + + +```python in with ua.set_backend(be1): print(myfunc()) ``` - Potato +--- + +```txt out +Potato +``` + + -``` python + + +```python in with ua.set_backend(be2): print(myfunc()) ``` - Strawberry +--- + +```text out +Strawberry +``` + + As we can clearly see: We have already provided a way to do (1) and (2) above. But then we run across the problem: How do we decide between @@ -119,23 +129,31 @@ these backends? How do we move between them? Let's go ahead and register both of these backends for permanent use. And see what happens when we want to implement both of their methods! -``` python +```python ua.register_backend(be1) ua.register_backend(be2) ``` -``` python + + +```python in print(myfunc()) ``` - Potato +--- + +```text out +Potato +``` + + As we see, we get only the first backend's answer. In general, it's indeterminate what backend will be selected. But, this is a special case: We're not passing arguments in! What if we change one of these to return `NotImplemented`? -``` python +```python # We redefine the multimethod so it's new again @ua.create_multimethod(myfunc_rd) def myfunc(): @@ -158,38 +176,54 @@ ua.register_backend(be1) ua.register_backend(be2) ``` -``` python + + +```python in with ua.set_backend(be1): print(myfunc()) ``` - Strawberry +--- + +```txt out +Strawberry +``` + + Wait\... What? Didn't we just set the first `Backend`? Ahh, but, you -see\... It's signalling that it has *no* implementation for `myfunc`. +see\... It's signalling that it has _no_ implementation for `myfunc`. The same would happen if you simply didn't register one. To force a `Backend`, we must use `only=True` or `coerce=True`, the difference will be explained in just a moment. -``` python + + +```python in with ua.set_backend(be1, only=True): print(myfunc()) ``` - --------------------------------------------------------------------------- - BackendNotImplementedError Traceback (most recent call last) - in - 1 with ua.set_backend(be1, only=True): - ----> 2 print(myfunc()) +--- - ~/Quansight/uarray/uarray/backend.py in __call__(self, *args, **kwargs) - 108 - 109 if result is NotImplemented: - --> 110 raise BackendNotImplementedError('No selected backends had an implementation for this method.') - 111 - 112 return result +```txt out +--------------------------------------------------------------------------- +BackendNotImplementedError Traceback (most recent call last) + in + 1 with ua.set_backend(be1, only=True): +----> 2 print(myfunc()) + +~/Quansight/uarray/uarray/backend.py in __call__(self, *args, **kwargs) + 108 + 109 if result is NotImplemented: +--> 110 raise BackendNotImplementedError('No selected backends had an implementation for this method.') + 111 + 112 return result + +BackendNotImplementedError: No selected backends had an implementation for this method. +``` - BackendNotImplementedError: No selected backends had an implementation for this method. + Now we are told that no backends had an implementation for this function (which is nice, good error messages are nice!) @@ -199,7 +233,7 @@ Now we are told that no backends had an implementation for this function Let's say we had two `Backend`s. Let's choose the completely useless example of one storing a number as an `int` and one as a `float`. -``` python +```python class Number(ua.DispatchableInstance): pass @@ -222,56 +256,82 @@ Number.register_convertor(be2, lambda x: str(x)) Let's also define a \"catch-all\" method. This catches all implementations of methods not already registered. -``` python +```python # This can be arbitrarily complex def gen_impl1(method, args, kwargs, dispatchable_args): if not all(isinstance(a, Number) and isinstance(a.value, int) for a in dispatchable_args): return NotImplemented - + return args[0] # This can be arbitrarily complex def gen_impl2(method, args, kwargs, dispatchable_args): if not all(isinstance(a, Number) and isinstance(a.value, str) for a in dispatchable_args): return NotImplemented - + return args[0] be1.register_implementation(None, gen_impl1) be2.register_implementation(None, gen_impl2) ``` -``` python + + +```python in myfunc('1') # This calls the second implementation ``` - '1' +--- + +```txt out +'1' +``` + + + + -``` python +```python in myfunc(1) # This calls the first implementation ``` - 1 +--- + +```txt out +1 +``` + + -``` python + + +```python in myfunc(1.0) # This fails ``` +--- + +```txt out --------------------------------------------------------------------------- BackendNotImplementedError Traceback (most recent call last) in ----> 1 myfunc(1.0) # This fails ~/Quansight/uarray/uarray/backend.py in __call__(self, *args, **kwargs) - 108 + 108 109 if result is NotImplemented: --> 110 raise BackendNotImplementedError('No selected backends had an implementation for this method.') - 111 + 111 112 return result BackendNotImplementedError: No selected backends had an implementation for this method. +``` -``` python + + + + +```python in # But works if we do this: with ua.set_backend(be1, coerce=True): @@ -281,8 +341,14 @@ with ua.set_backend(be2, coerce=True): print(type(myfunc(1.0))) ``` - - +--- + +```txt out + + +``` + + This may seem like too much work, but remember that it's broken down into a lot of small steps: @@ -305,7 +371,9 @@ right values from it. `unumpy` is a set of NumPy-related multimethods built on top of `uarray`. You can use them as follows: -``` python + + +```python in import unumpy as np # Note the changed import statement from unumpy.xnd_backend import XndBackend @@ -313,7 +381,13 @@ with ua.set_backend(XndBackend): print(type(np.arange(0, 100, 1))) ``` - +--- + +```txt out + +``` + + And, as you can see, we get back an Xnd array when using a NumPy-like API. Currently, there are three back-ends: NumPy, Xnd and PyTorch. The @@ -324,8 +398,7 @@ We are also working on supporting more of the NumPy API, and dispatching over dtypes. Feel free to browse the source and open issues at: - or shoot me an email at -``{=html}habbasi@quansight.com``{=html} -if you want to contact me directly. You can also find the full -documentation at . +https://github.com/Quansight-Labs/uarray or shoot me an email at +habbasi@quansight.com +if you want to contact me directly. You can also find the full documentation at https://uarray.readthedocs.io/en/latest/.