Argument parsing for functions

[emmatyping]

To make writing Rust functions that take Python arguments ergonomic, we need an easy way to describe how to parse arguments, and ideally convert to Rust object abstractions. The C API for argument parsing is described here: https://docs.python.org/3/c-api/arg.html

C code in CPython uses a DSL to pre-generate C argument parsing code called argument clinic

We have a few different ways we could go here. PyO3 currently uses proc-macros to provide very ergonomic argument parsing. We could also modify argument clinic to generate Rust code and include!() the generated files, which might have better compilation times.

I am still undecided about which approach I prefer.

[youknowone]

Maybe using clinic + proc-macro will is also possible.

#[clinic]
/// useful DSL
fn xyz_impl(...) 
``

Then #[clinic] can generate everything inside the macro. No `include!()` will be required.
Cons: The generated code will not be searched 

If saving generated code in the repository is important, generate the code and including it using `#[clinic]` will be also possible though.

[davidhewitt]

I have no strong opinion about the build system integration, anything is possible. I think what matters most is going to be what the Rust code we write looks like. Does it use normal Rust attributes, or something more like argument clinic special syntax?

[Eclips4]

I’m pretty much in favor of any approach except updating the Tools/clinic one. The basic problem is that CPython's clinic isn't guaranteed to work with third-party C modules. I think we can use a proc-macro instead, so we could make it usable for third-party modules that way.

[emmatyping]

I think we can use a proc-macro instead, so we could make it usable for third-party modules that way.

Yeah I think this is a good idea. The APIs we design now should eventually be stablized and made usable by the public, so proc-macros seem like a natural use case for something we'll want to make ergonomic for users. It also makes a lot of sense as we will likely be iterating on function signatures initially a lot.

If compile times are an issue, we can probably re-use the proc-macro code to generate argument parsing code that gets checked in and used via a build script hook (similar to how bindgen can generate bindings ahead of time).