✨: HasX attributes #34

nstarman · 2025-07-01T21:18:00Z

~~Requires #32. I'll rebase when that's in.~~

Now preceding #32.

nstarman · 2025-07-01T21:19:35Z

@jorenham @NeilGirdhar @lucascolley this doesn't resolve the current discussion re a DTypeT typevar or a DType protocol, but does mean Arrays can now understand any typevar or protocol we want to add.

jorenham

I like the Has* protocols. Optype will cover the CanArray* ones I think. There might be some that can't be expressed because Self can't be passed as generic type argument, but I suppose we can deal with it when needed.

One nit is that ... are not needed when there's a docstring, which already counts as an expression or statement or something.

nstarman · 2025-07-02T21:37:08Z

I like the Has* protocols. Optype will cover the CanArray* ones I think. There might be some that can't be expressed because Self can't be passed as generic type argument, but I suppose we can deal with it when needed.

What do we do about docstrings?

One nit is that ... are not needed when there's a docstring, which already counts as an expression or statement or something.

Yes, my pylint is complaining. But IMO empty methods should have ... to distinguish them from ones that aren't empty. Helps in understanding inheritance with a Protocol.

jorenham · 2025-07-02T21:42:40Z

What do we do about docstrings?

Hmm good question. Maybe a numpy-esque add_docstring function?

jorenham · 2025-07-02T21:44:49Z

Yes, my pylint is complaining. But IMO empty methods should have ... to distinguish them from ones that aren't empty. Helps in understanding inheritance with a Protocol.

Yea I guess there's something to be said for that. It just looks a bit weird to me to see a ... occupy a line on its own (but that might have something to do with me spending most of my breathing time looking at stubs).

src/array_api_typing/_array.py

nstarman · 2025-07-24T02:00:41Z

@jorenham I added tests and made the protocols public.

nstarman · 2025-07-27T03:21:06Z

@jorenham This should cover all the array attributes.

nstarman · 2025-08-01T17:30:10Z

@jorenham it might be easier to merge this before doing numpy type compat stuff from #32.

src/array_api_typing/_array.py

jorenham · 2025-08-05T15:31:59Z

src/array_api_typing/_array.py

+        ...
+
+
+class HasSize(Protocol):


What's the use-case of this one? Is there anything this can help with, that HasShape can't?
Put differently; should we make this public API or not?

Shouldn't we have Protocols for all attributes in the Array API?

I don't see why we should write a protocol if there's no use for it.

def is_sized(obj: Any, /) -> TypeGuard[HasSize]: ...

ok, and what would you use that for in a real-world scenario

But I'm trying to prevent is situations where there are multiple non-obvious ways of achieving the same result. And HasSize seems to me like it could have a lot of overlap with HasShape.

This library isn't the venue for changing the Array API...

I know, and I wasn't suggesting anything like that.

Is that not the practical effect of omission?

If the Array API spec has methods size and shape, and the corresponding typing library has only HasShape then by omission it re-interprets the Array API. Yes people can write their own HasSize, but the target audience of the Array API isn't just lovers of typing.

Is that not the practical effect of omission?

No? AFAIK, this is library depends on the array api, not the other way around. But maybe I'm being naive here 🤷🏻.

Plus, there are already stub-like types in the array-api itself (https://github.com/data-apis/array-api/tree/main/src/array_api_stubs) that are used for documentation purposes (and don't have any use-case besides that).

They aren't planning on publishing those. As I understand it, we depend on the Array API spec and our remit is to describe it statically. As the official typing library for that spec, omission is change.

We have HasNamespace. We should have similar for the other methods and attributes of Array. We should have useful intersection types related to Array. We should have DoesFunction protocols for the functions and a Namespace protocol for their collection.

src/array_api_typing/_array.py

jorenham · 2025-08-05T15:34:30Z

src/array_api_typing/_array.py

+    HasNDim,
+    HasShape,
+    HasSize,
+    HasTranspose,


This HasTranspose restricts this to 2d arrays

Does it? I don't know of any array object that doesn't have this method, regardless of the value's dimensionality.

.transpose() is only defined for 2d arrays, so we should expect array libs to follow that, and we should therefore expect there to be annotations like

class SomeArray[ShapeT, DTypeT]: # --snip-- def transpose(self: SomeArray[Rank2, DTypeT]) -> SomeArray[Rank2, DTypeT]: ...

So a SomeArray[Rank1] won't be assignable to xpt.Array.

That sounds great.
It sounds like there are two interpretations of how this should be represented statically given the API spec: "The array instance must be two-dimensional. If the array instance is not two-dimensional, an error should be raised."

That it statically determines the shape. This focuses on the 1st sentence.

@property def T(self: SomeArray[Rank2, DTypeT]) -> SomeArray[Rank2, DTypeT]: ...

that it shadow the runtime. This focuses on the 2nd sentence.

@overload @property def T(self: SomeArray[Rank0 | Rank1, DTypeT]) -> Never: ... @overload @property def T(self: SomeArray[Rank2, DTypeT]) -> SomeArray[Rank2, DTypeT]: ... @overload @property def T(self: SomeArray[RankGE3, DTypeT]) -> Never: ...

(or if we don't have shape typing)

def T(self: SomeArray) -> SomeArray | Never: ... # yes, the Never is ignored.

This was written more with the 2nd interpretation in mind, but I'm very happy to accept the 1st interpretation and remove T from the default Array object, so long as we still include some form of HasTranspose Protocol for people to use.
I do think there is then interesting discussion about that Protocol ....
Should we vendor 2 forms? one for each interpretation? That will help people when building their own intersection types.

tests/integration/test_numpy2p0.pyi

src/array_api_typing/_array.py

jorenham · 2025-08-05T15:43:48Z

src/array_api_typing/_array.py

+        ...
+
+
+class HasNDim(Protocol):


Is there a situation where you'd wanna use HasNDim over e.g. HasShape? Otherwise we probably should keep this private, given that

There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.

If you're checking for the attribute ndim.
Not sure I'm understanding this comment. Ndim is in the Array API spec...

Well, my point is that I think we should only provide users with protocols that help them annotate their array-api code. Otherwise, it'll just be confusing for the users, and a waste of time for us.

Oh. As the Array API is the intersection of the array libraries, IMO pretty much everything is useful.

Yea of course, everything in the array-api is designed for a reason. But all of those reasons apply in the runtime world. And our goal is to provide an API for the static-typing world, which is but a shadow of the runtime one, where only a subset of the API has practical use.

Python comes with batteries.
The Array API is the intersection of the array libraries and each method is useful / used by many people / has long historical precedent, which is why it's in all the libraries.
IMO this library isn't to ideologically improve on the Array API, it's to provide the tools to describe it statically.

My suggestion is to have all these low-level protocols. Where we go beyond the baseline Array API is in how these protocols are parametrized and how we build intersection types. This is where necessity drives us to make decisions.

But the main difference is that almost all of the optype protocols were written with a specific use-case in mind

I don't disagree with this idea, but let's consider the situation. There are numerous array libraries which people have spent a long time working on, thinking about their APIs for many years. Numpy in particular sets much of the standard and has many people thinking about the things in it. They got together to rethink numpy and array libraries in general, to mass adoption. We've been to some of those meetings where they're agonizing over a small piece of API. IMO the presumption of usefulness is very clear.

@jorenham can we get in protocols for the methods?

#22 (comment)

@jorenham can we get in protocols for the methods?

#22 (comment)

What methods, exactly?

Methods and attributes of Array.

src/array_api_typing/_array.py

Signed-off-by: nstarman <[email protected]>

nstarman · 2025-08-22T15:36:47Z

@jorenham. I can see your points about some of the dangers of building the Array intersection protocol from all these method-by-method protocols before we've figured out shape typing, etc.
If we can agree to have method-by-method Protocols for https://data-apis.org/array-api/2021.12/API_specification/index.html#api-specification-index--page-root then we can pause on making Array while we more quickly build out the underlying protocols.
I imagine there will be some revisions to the individual Protocols necessary when stitching them together into various Array-like protocols that we didn't see when writing them individually, but I think these are separable steps and concerns and we should fill out the namespace ASAP.

nstarman · 2025-09-03T13:51:08Z

@jorenham what do you think?

nstarman · 2025-09-03T14:17:22Z

I'm removing the atomic protocols from Array and reworking the tests accordingly.

Signed-off-by: nstarman <[email protected]>

Signed-off-by: Nathaniel Starkman <[email protected]>

Signed-off-by: nstarman <[email protected]>

nstarman · 2025-09-10T00:05:37Z

@jorenham what do you think?

nstarman · 2025-09-20T18:28:27Z

Ping @jorenham

jorenham reviewed Jul 2, 2025

View reviewed changes

nstarman force-pushed the has_x_attributes branch 2 times, most recently from 8096aff to 3793c1f Compare July 21, 2025 15:56

nstarman mentioned this pull request Jul 22, 2025

✨: add CanArrayX protocols #32

Draft

nstarman force-pushed the has_x_attributes branch 5 times, most recently from cb3dbb3 to dec1842 Compare July 23, 2025 21:05

nstarman marked this pull request as ready for review July 23, 2025 21:10

nstarman requested a review from jorenham July 23, 2025 21:10

nstarman force-pushed the has_x_attributes branch from dec1842 to 53ddb98 Compare July 23, 2025 21:14

nstarman commented Jul 23, 2025

View reviewed changes

src/array_api_typing/_array.py Outdated Show resolved Hide resolved

nstarman force-pushed the has_x_attributes branch from 53ddb98 to a5e3d5b Compare July 24, 2025 01:57

nstarman added this to the v2021-12-0.0 milestone Aug 1, 2025

nstarman added ✨ feature Introduce new features. ✅ tests Add, update, or pass tests. labels Aug 1, 2025

nstarman commented Aug 1, 2025

View reviewed changes

src/array_api_typing/_array.py Show resolved Hide resolved

jorenham reviewed Aug 5, 2025

View reviewed changes

nstarman force-pushed the has_x_attributes branch from a5e3d5b to 27ef62d Compare August 20, 2025 01:16

💄rearrange protocols in Array

3cb0930

Signed-off-by: nstarman <[email protected]>

nstarman force-pushed the has_x_attributes branch from 27ef62d to c65c5d5 Compare August 20, 2025 01:22

nstarman requested a review from jorenham August 20, 2025 01:26

nstarman mentioned this pull request Aug 20, 2025

HasX Device Parametrization #52

Draft

nstarman mentioned this pull request Aug 22, 2025

✨start ArrayNamespace #56

Draft

nstarman changed the title ~~feat: HasX attributes~~ ✨: HasX attributes Aug 22, 2025

nstarman force-pushed the has_x_attributes branch from c65c5d5 to a45816c Compare September 3, 2025 14:16

nstarman force-pushed the has_x_attributes branch from a45816c to c23b17a Compare September 3, 2025 14:23

nstarman added 6 commits September 3, 2025 10:27

✨ HasDevice

d786599

Signed-off-by: nstarman <[email protected]>

✨ HasMatrixTranspose

4c3963b

Signed-off-by: nstarman <[email protected]>

✨ HasNDim

58698bb

Signed-off-by: nstarman <[email protected]>

✨ HasShape

25db39c

✨ HasSize

7f1eb4e

Signed-off-by: Nathaniel Starkman <[email protected]>

✨ HasTranspose

a53c176

Signed-off-by: nstarman <[email protected]>

nstarman force-pushed the has_x_attributes branch from c23b17a to a53c176 Compare September 3, 2025 14:27

✨: HasX attributes #34

Are you sure you want to change the base?

✨: HasX attributes #34

Conversation

nstarman commented Jul 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nstarman commented Jul 1, 2025

Uh oh!

jorenham left a comment

Choose a reason for hiding this comment

Uh oh!

nstarman commented Jul 2, 2025

Uh oh!

jorenham commented Jul 2, 2025

Uh oh!

jorenham commented Jul 2, 2025

Uh oh!

Uh oh!

nstarman commented Jul 24, 2025

Uh oh!

nstarman commented Jul 27, 2025

Uh oh!

nstarman commented Aug 1, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jorenham Oct 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nstarman Oct 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nstarman Aug 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nstarman Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nstarman Aug 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jorenham Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nstarman commented Jul 1, 2025 •

edited

Loading

jorenham Oct 3, 2025 •

edited

Loading

nstarman Oct 3, 2025 •

edited

Loading

nstarman Aug 18, 2025 •

edited

Loading

nstarman Aug 22, 2025 •

edited

Loading

nstarman Aug 18, 2025 •

edited

Loading

jorenham Aug 22, 2025 •

edited

Loading

nstarman Sep 23, 2025 •

edited

Loading

nstarman Sep 23, 2025 •

edited

Loading

nstarman commented Aug 22, 2025 •

edited

Loading