32-bit support?

[oxyCabhru]

I'd like to work with willow on something like an esp32, which feels like fighting with a bunch of eels trying to electrocute me and everyone I have ever loved
Will there be 32-bit support, such that compact_u64, ufotofu could gate x64 behind #[cfg(target_has_atomic = "64")] ?
Posting this here since the main willow_data_model makes extensive use of compact_u64, so simply stubbing it out isn't very feasible
cheers~

[AljoschaMeyer]

Hi @oxyCabhru ,

cool to see you interested in this - we had microcontrollers very much on our mind when designing the specs (less so when writing the current rust implementation, to be honest). Sorry to hear that these eels are bothering you and your loved ones.

Can you make a bit more precise what you are asking for? By 32-bit support, do you mean a version of the Willow specs that works with 32-bit ints instead of 64 bit ints? That would be tricky to impossible (with our resources), and not entirely desirable because it would reduce interoperability. Or do you mean rust implementations which work with 32 bit ints instead of 64 bit ints and panic (or report errors) whenever a user or peer supplies data that goes beyond the resulting limits?

I'm unfamiliar with the consequences of using rust on 32-bit platforms. Is u64 available, or is it only atomic 64-bit ints which are missing? Off the top of my head, I don't think we use atomic 64-bit ints anywhere, since our code is async but intended for single-threaded executors.

Note that ufotofu itself should be completely agnostic to 32 vs 64 bit; the crates that would be affected here are probably compact_u64 and everything which depends on it.

[oxyCabhru]

Hi hello yes the eels are very difficult to reason with.
I'm still a novice when it comes to.. everything, so bear with me :D
My use case would be utilizing willow as a protocol of communication across esp32-based devices, which obviously limits the use of 64-bit ints
Since these devices won't be interacting with any 64-bit based devices, I don't think a version of willow that is 32-bit only is needed, just a rust implementation that errors out when it comes to the unexpected 64-bit data -- just as suggested yes
I've forked wrapper-rs to include a flag that compiles AtomicU64 and AtomicI64 only if the target supports it, as well as ufotofu_condec_endian to utilize this flag, and yes, now compact_u64 is the culprit as it simply has to be stubbed out entirely
I spent a night trying to gate 64 atomics behind a flag, alas my prowess lacks elegance
I must've done something wrong along the way, but essentially I used esp-idf-template to setup the project, added willow_data_model, meadowcap, and willow_25, and it just won't really compile:

   Compiling futures-util v0.3.31
   Compiling typenum v1.18.0
   Compiling pin-project v1.1.10
   Compiling arbitrary v1.4.2
   Compiling zerocopy v0.8.26
   Compiling wrapper v0.1.1
   Compiling getrandom v0.2.16
   Compiling ufotofu_queues v0.5.0
error[E0412]: cannot find type `AtomicI64` in module `core::sync::atomic`
    --> ~/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wrapper-0.1.1/src/impls_core.rs:55:43
     |
55   |   implement_into_inner!(core::sync::atomic::AtomicI64, i64);
     |                                             ^^^^^^^^^ help: a struct with a similar name exists: `AtomicI16`
     |
    ::: ~/.rustup/toolchains/esp/lib/rustlib/src/rust/library/core/src/sync/atomic.rs:3573:1
     |
3573 | / atomic_int! {
3574 | |     cfg(target_has_atomic = "16"),
3575 | |     cfg(target_has_atomic_equal_alignment = "16"),
3576 | |     stable(feature = "integer_atomics_stable", since = "1.34.0"),
...    |
3589 | |     i16 AtomicI16
3590 | | }
     | |_- similarly named struct `AtomicI16` defined here

error[E0412]: cannot find type `AtomicU64` in module `core::sync::atomic`
    --> ~/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wrapper-0.1.1/src/impls_core.rs:60:43
     |
60   |   implement_into_inner!(core::sync::atomic::AtomicU64, u64);
     |                                             ^^^^^^^^^ help: a struct with a similar name exists: `AtomicU16`
     |
    ::: ~/.rustup/toolchains/esp/lib/rustlib/src/rust/library/core/src/sync/atomic.rs:3592:1
     |
3592 | / atomic_int! {
3593 | |     cfg(target_has_atomic = "16"),
3594 | |     cfg(target_has_atomic_equal_alignment = "16"),
3595 | |     stable(feature = "integer_atomics_stable", since = "1.34.0"),
...    |
3608 | |     u16 AtomicU16
3609 | | }
     | |_- similarly named struct `AtomicU16` defined here

For more information about this error, try `rustc --explain E0412`.
error: could not compile `wrapper` (lib) due to 2 previous errors
warning: build failed, waiting for other jobs to finish...

cargo build output, this is done without the stupid patches i made

Hopefully this is enough to go by, I bit off way more than I can chew :')

[AljoschaMeyer]

So... our current development versions of the willow crates (not yet released to crates.io) do not depend on the wrapper crate any longer. So this is a problem that will fix itself automatically for you.

You could play around with depending on this repository directly instead of using the versions on crates.io, and possibly use the [patch] feature of cargo to get what you need.

An alternative to get things running for now could be for you to fork the wrapper crate, simply rip out everything related to atomic 64-bit integers, and then use the [patch] feature of cargo to insert your forked version everywhere.

Nothing in the willow crates actually relies on those atomic 64-bit integers; this is a rather painfully pointless hiccup the eels have been sending your way =(

[oxyCabhru]

After patching my project to use the crates in the current main branch, still no dice as ufotofu_codec_endian refuses to compile:

error[E0276]: impl has stricter requirements than trait
impl has extra requirement `C: BulkConsumer`
impl has extra requirement `P: BulkProducer`
impl has extra requirement `C: ufotofu::Consumer`
impl has extra requirement `P: ufotofu::Producer`

x144
full build log here

These conniving little eels must not get the best of us!
(i'm sorry)

[AljoschaMeyer]

I am sorry, this is only so annoying because we have split our code into too many crates, and those crates have had too many breaking changes. I've been fighting similar issues quite often as well... but this particular problem is sadly not familiar.

Our dependencies on the main branch might be a bit wonky, do things improve if you depend on the rbsr branch of this repo? If that doesn't fix things, I'd need the output of cargo tree to help you troubleshoot further.