Common string parser

[andrewgsavage]

Should the API provide a common parser to convert strings to a dict-like object for the libraries to use? I think this is necessary to ensure a consistency between libraries whch may use slightly different rules for parsing.

m / kg s could interpreted as m^11 kg^-1 s^-1 or m^11 kg^-1 s^1

[neutrinoceros]

m / kg s could interpreted as m^11 kg^-1 s^-1 or m^11 kg^-1 s^1

To avoid this ambiguity, maybe we should avoid guess work entirely and never infer the operation implied by whitespace ?

[andrewgsavage]

That's one example and yes we could provide tests to specify how the parser should behave. However I expect there to be many subtlities that would need specifying.

The pint parser is different to the astropy parser so it's not guaranteed a string would be parsed the same way. For code using the API to be compatable with multiple libraries, it's important for a string to be parsed into the same units.

We could have code in each unit library for the parser that passes tests given by the API, or we could include a parser. I think including a parser requires less work overall.

[nstarman]

Going to quote from https://github.com/quantity-dev/dimension-api/issues/3 which discussed a common parser for dimensions. But the ideas are similar to units.

If the various libraries want to implement a common schema / parser that would a wonderful thing to do! E.g.

# library common_dimension_parser.py

def parse_str(str) -> dict[str, int]:
    """e.g. [length]/[time] -> {"[length]": 1, "[time]": -1}"""

Now implementing libraries can make common_dimension_parser a hard dependency and use the dict[str, int] to construct their library-specific implementation of Dimension. E.g. pint's would be

...
parsed = parse_str(...)
return UnitsContainer(parsed)

While Astropy's might be

return functools.reduce(multiply, [Dimension(k) ** v for k, v in parse_str(...).items()])

So yes, a common string parser would be a good library. Here's why it shouldn't be baked into an API library: this is entirely about implementation, not behaviour. E.g. Astropy implements a number of different unit parsers because there are different standards. But all units, once initialized behave the same way because they are following Astropy's API.

If we look to https://data-apis.org/array-api/latest/API_specification/data_types.html, we see that while they discuss what are the minimum required set of dtypes, there is no object to actually describe these data types, no way to initialize them, no way to parse them. It's purely discussion, but constitutes expectations for conforming to the API. Bringing this back to units, the equivalent would be if we enumerated in the API docs what are the minimum set of units (and the strings which can be used to initialize them) (we probably should do this). But again, no implementation! That's left to implementing libraries.

We can, as this discussion suggestions and of which I am a majorly in favor of, have a common parser library that helps implement these API requirements. This is an implementation of part of the API and one which implementing libraries can choose to have as a runtime dependency. It is not, however, baked into the API itself.

Really all I'm saying is 💯🎉 to the idea. 👎 to accessing the parser as from api import parser instead of from common_parser import parser.

[neutrinoceros]

Here's why it shouldn't be baked into an API library: this is entirely about implementation, not behaviour.

I strongly agree with this.

It is sounding more and more like we really want our final product to be a collection of "interoperable"¹ specs and packages, and the more I think about it, the more a monorepo/workspace workflow seems like the way to assemble all moving parts into a high-cohesion whole.

Footnotes

1. for lack of a better term, even though it doesn't feel very appropriate for API specifications ↩