Reusing Rust type bindings / support for type aliases in the Rust section

[anforowicz]

PTAL?

This PR fixes https://crbug.com/433254489 (/cc @ShabbyX as FYI - IIUC they've patched this PR onto their local machine and verified that it does indeed unblock them).

The first commit fixes some existing code duplication and prevents more code duplication around how fn syntax::parse_items consumes a syntax::Module. I think this commit is okay and moves things in a reasonable direction, but I acknowledge that adding more partial consumption/destructuring/taking is a bit icky. More discussion about this aspect of the changes can be found in a comment from a semi-internal review at anforowicz#2 (comment) and anforowicz#2 (comment).

I also note that some existing variants of syntax::Api enum have Rust and C++ versions - e.g. RustFunction and CxxFunction as well as RustType and CxxType. We could also do that for the TypeAlias variant which this PR doesn't touch just yet. Please provide feedback on whether I should also submit an additional commit into this or a separate PR - see https://github.com/anforowicz/cxx/tree/api-rust-type-alias-separate-variant

/cc @zetafunction who has kindly provided initial, semi-internal feedback at anforowicz#2

[dtolnay]

Reusing an existing defined pair of C++ and Rust type is already supported. I don't understand the distinction between the extern C++ type aliases which are already supported, and the extern Rust type aliases in this PR. What is the situation where it would be correct to write extern "Rust" { type T = path::to::U; } and not extern "C++" { type T = path::to::U; }?

[anforowicz]

Reusing an existing defined pair of C++ and Rust type is already supported. I don't understand the distinction between the extern C++ type aliases which are already supported, and the extern Rust type aliases in this PR. What is the situation where it would be correct to write extern "Rust" { type T = path::to::U; } and not extern "C++" { type T = path::to::U; }?

In the PR, in the end-to-end test under tests/ffi/... the CrossModuleRustType is a Rust type, rather than a C++ type. This is why extern "C++" { type T = path::to::U; } will not work and we need to support extern "Rust" { type T = path::to::U; }.

Let me consider different scenarios:

• If Foo is a C++ type (declared in another #[cxx::bridge]'s extern "C++" section) then extern "C++" { type Foo = crate::some::other::module::Foo; } will indeed work.
• If Foo is a Rust type (declared in another Rust module, and also covered by another #[cxx::bridge]) then:
  • extern "C++" { type Foo = crate::some::other::module::Foo; } will not work, because Foo is a Rust type rather than a C++ type/binding. In particular, if I change tests/ffi/lib.rs in the PR to put the type alias in an extern "C++" section, then cargo test rightfully fails saying: "the trait bound module::CrossModuleRustType: ExternType is not satisfied".
  • I can't just say extern "Rust" { type Foo; /* ... */ } and have type Foo = crate::some::other::module::Foo outside of cxx::bridge, because this will error out with: "conflicting implementations of trait RustType for type module::CrossModuleRustType" (or Foo, this error message was captured by tweaking the end-to-end tests from the PR).

[dtolnay]

This is not sufficient for safety. Consider adding:

        // inside extern "Rust"
        type Any = crate::module::CrossModuleRustType;
        fn repro(bad: &mut CrossModuleRustType) -> &mut Any;
...

fn repro(bad: &mut ffi::CrossModuleRustType) -> &mut ffi::Any {
    bad
}

Nothing here enforces what type the C++ Any is. We can make it anything:

namespace tests {
using Any = std::array<char, 1000>;
}

and arbitrarily stomp on memory.

repro(*r_boxed_cross_module_rust_type(123)).fill('?');

     Running tests/test.rs (target/debug/deps/test-f2f5723622a08a56)

running 15 tests
free(): invalid next size (fast)
malloc(): corrupted top size
error: test failed, to rerun pass `--test test`

Caused by:
  process didn't exit successfully: `target/debug/deps/test-f2f5723622a08a56` (signal: 6, SIGABRT: process abort signal)

[anforowicz]

I don't understand why you say that Any is a C++ type. It is a Rust type, right? (by virtue of ::cxx::private::verify_rust_type which ensures that T: RustType which means the aliased type must have been used in another cxx::bridge's extern "Rust" section.

[anforowicz]

In particular, if I change the new tests/ffi/lib.rs snippet from the PR to:

extern "Rust" {
    type CrossModuleRustType = crate::ffi::C;     // `C` is a C++ type - it won't work here.
    fn r_get_value_from_cross_module_rust_type(
        value: &CrossModuleRustType,              
    ) -> i32;
}                                                 

then cargo test expectedly reports an error that says this is disallowed: "the trait bound ffi::C: RustType is not satisfied".

---

Maybe I just have trouble figuring out what scenario you have in mind. If the error above (i.e. "free(): invalid next size" and "malloc(): corrupted top size") can be reproed, then I would really appreciate if you could share the repro as a commit on top/after my PR?

[dtolnay]

If the error above (i.e. "free(): invalid next size" and "malloc(): corrupted top size") can be reproed, then I would really appreciate if you could share the repro as a commit on top/after my PR?

This is what I mean: anforowicz#3. This would need to not compile in order for the implementation to be sound.

[anforowicz]

Thanks! That helped. I understand the problem now.

The specific scenario that you've pointed out should now be fixed in the latest version of the PR. The fix is to check on C++ side that the type alias is derived from ::rust::Opaque. OTOH, the fix doesn't seem sufficient to solve the general problem - transmuting into an unrelated type seems still posslble with something like:

#[cxx::bridge]
mod ffi {
    extern "Rust" {
        type RustType;
        type OtherRustType;

        #[cxx_name = "OtherRustType"]
        type RustAlias = RustType;

        fn bad_transmute(x: &RustType) -> &RustAlias;
        fn do_something_with_other_rust_type(x: &OtherRustType);
    }
}

To prevent the above, we'd need to assert on C++ side not only that RustAlias aliases an opaque Rust type, but that it aliases the same, specific Rust type. In other words, this probably requires a C++ equivalent of ExternType::Id checks from Rust side.

Q1: Does the above sound more or less correct to you?

Q2: Do you have suggestions on how to implement the C++ equivalent of ExternType::Id checks? Some initial thoughts below:

• It seems that the type id of a Rust type should include the type name, the module path, and the crate name. The latter seems difficult, because 1) IIUC cxxbridge does not currently know/get the crate name, and 2) crate names can be aliased.
• One mechanism to add a mechanism for type id checks would be to tweak write_opaque_type so that the generated struct has an extra field like so:

#include <cstring>

struct S /* : ::rust::Opaque */ {
    static constexpr const char* __cxxbridge1_rust_type_id = "module::path::RustType";
};

static_assert(
    0 == strcmp(S::__cxxbridge1_rust_type_id, "module::path::OtherRustType"),
    "blah"
);

(having a new public field is a bit icky but is not the end of the world; one alternative would be to introduce template parameters into ::rust::Opaque, but changing this type should probably be treated as a breaking change which seems to make this approach undesirable)

[dtolnay]

I still feel that all the use cases of this functionality are already well supported, with all the static checking required for soundness, by extern "C++". An extern "C++" type alias means "I already have a C++ definition of a type (the left-hand side) and a Rust definition of a type (right-hand side) that refer to the same type". This is verified in Rust via ExternType::Id and redoing an equivalent check on the C++ side is unnecessary.

If I wipe out all the changes in {gen,macro,src,syntax} from this PR, but keep the changes in tests/, with this conversion to extern "C++" the new tests pass:

--- a/tests/ffi/lib.rs
+++ b/tests/ffi/lib.rs
@@ -370,8 +370,11 @@ pub mod ffi {
     impl SharedPtr<Private> {}
     impl UniquePtr<Array> {}
 
-    extern "Rust" {
+    extern "C++" {
         type CrossModuleRustType = crate::module::CrossModuleRustType;
+    }
+
+    extern "Rust" {
         fn r_get_value_from_cross_module_rust_type(value: &CrossModuleRustType) -> i32;
     }
 }
--- a/tests/ffi/module.rs
+++ b/tests/ffi/module.rs
@@ -82,6 +82,7 @@ pub mod ffi2 {
 #[cxx::bridge(namespace = "tests")]
 pub mod ffi3 {
     extern "Rust" {
+        #[derive(ExternType)]
         type CrossModuleRustType;
 
         #[allow(clippy::unnecessary_box_returns)]

In 2.x it would be a good idea to provide ExternType impls for every opaque Rust type automatically, but this is a breaking change because they conflict with handwritten impls which may currently exist.

Other than that I remain unsure what is left to implement. Would it be less confusing if type aliases are placed at the module level outside of extern? Or if RustType is changed to carry an Id associated type (and potentially made public)?

[ShabbyX]

@dtolnay (regarding #1539 (comment) so I don't intrude on that thread), the use case I have that doesn't work with existing cxx is this:

• I have a(n old) C++ library, it's a part of a bigger project
• I'm replacing parts of it with a (new) Rust implementation. Parts of it, because it's a big project that I need to replace piecemeal
• In Rust, I have a type that represents the main object of the library: struct IR
• I need to pass this Rust object (in a Box) to C++ (say, in phase 1 (that is parse)), so cxx in one file defines extern Rust { type IR; }
• I need to receive this Rust object from C++ (say, in phase 2 (that is codegen)), so cxx in another file defines extern Rust { type IR; }

Note again that IR is a Rust type.

This results in a build error:

error[E0119]: conflicting implementations of trait `RustType` for type `ir::IR`
    --> ../../src/compiler/translator/ir/src/compile.rs:54:9
     |
54   |         type IR;
     |         ^^^^^^^ conflicting implementation for `ir::IR`
     |
    ::: ../../src/compiler/translator/ir/src/builder.rs:2949:9
     |
2949 |         type IR;
     |         ------- first implementation here

error[E0119]: conflicting implementations of trait `ImplBox` for type `ir::IR`
    --> ../../src/compiler/translator/ir/src/compile.rs:59:17
     |
59   |             ir: Box<IR>,
     |                 ^^^^^^ conflicting implementation for `ir::IR`
     |
    ::: ../../src/compiler/translator/ir/src/builder.rs:2952:64
     |
2952 |         fn builder_finish(mut builder: Box<BuilderWrapper>) -> Box<IR>;
     |                                                                ------ first implementation here

If I attempt to alias them with type IR = crate::builder::ffi::IR;, I get:

error[cxxbridge]: type alias in extern "Rust" block is not supported
   ┌─ ../../src/compiler/translator/ir/src/compile.rs:56:9
   │
56 │         type IR = crate::builder::ffi::IR;
   │         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ type alias in extern "Rust" block is not supported

So in response to "Other than that I remain unsure what is left to implement", do you have a suggestion on how to support this use case? Ideally, I wouldn't have to put all the (huge) cxx code from multiple files into one as a workaround.