Partial conversion to CppRef<T> everywhere

Background:

Rust references have certain rules, most notably that the underlying data
cannot be changed while an immutable reference exists. That's essentially
impossible to promise for any C++ data; C++ may retain references or pointers
to data any modify it at any time. This presents a problem for Rust/C++ interop
tooling. Various solutions or workarounds are possible:

1) All C++ data is represented as zero-sized types.
   This is the approach taken by cxx for opaque types. This sidesteps all of the
   Rust reference rules, since those rules only apply to areas of memory that
   are referred to. This doesn't really work well enough for autocxx since
   we want to be able to keep C++ data on the Rust stack, using all the fancy
   moveit shenanigans, and that means that Rust must know the true size and
   alignment of the type.

2) All C++ data is represented as UnsafeCell<MaybeUninit<T>>.
   This also sidesteps the reference rules. This would be a valid option for
   autocxx.

3) Have a sufficiently simple language boundary that humans can
   reasonably guarantee there are no outstanding references on the C++ side
   which could be used to modify the underlying data.
   This is the approach taken by cxx for cxx::kind::Trivial types. It's
   just about possible to cause UB using one of these types, but you really
   have to work at it. In practice such UB is unlikely.

4) Never allow Rust references to C++ types. Instead use a special
   smart pointer type in Rust, representing a C++ reference.
   This is the direction in this PR.

More detail on this last approach here:
https://medium.com/@adetaylor/are-we-reference-yet-c-references-in-rust-72c1c6c7015a

This facility is already in autocxx, by adopting the safety policy
"unsafe_references_wrapped". However, it hasn't really been battle tested
and has a bunch of deficiencies.

It's been awaiting formal Rust support for "arbitrary self types" so that methods
can be called on such smart pointers. This is now
[fairly close to stabilization](rust-lang/rust#44874 (comment));
this PR is part of the experimentation required to investigate whether that rustc
feature should go ahead and get stabilized.

This PR essentially converts autocxx to only operate in this mode - there should
no longer ever be Rust references to C++ data.

This PR is incomplete:
* There are still 60 failing integration tests. Mostly these relate to subclass
  support, which isn't yet converted.
* `ValueParam` and `RValueParam` need to support taking `CppPin<T>`, and possibly
  `CppRef<T: CopyNew>` etc.
* Because we can't implement `Deref` for `cxx::UniquePtr<T>` to emit a
  `CppRef<T>`, unfortunately `cxx::UniquePtr<T>` can't be used in cases where
  we want to provide a `const T&`. It's necessary to call `.as_cpp_ref()` on the
  `UniquePtr`. This is sufficiently annoying that it might be necessary to
  implement a trait `ReferenceParam` like we have for `ValueParam` and `RValueParam`.
  (Alternatives include upstreaming `CppRef<T>` into cxx, but per reason 4 listed
  above, the complexity probably isn't merited for statically-declared cxx
  interfaces; or separating from cxx entirely.)

This also shows up a [Rustc problem which is fixed here](rust-lang/rust#135179).

Ergonomic findings:
* The problem with `cxx::UniquePtr` as noted above.
* It's nice that `Deref` coercion allows methods to be called on `CppPin` as well
  as `CppRef`.
* To get the same benefit for parameters being passed in by reference, you need
  to pass in `&my_cpp_pin_wrapped_thing` which is weird given that the whole
  point is we're trying to avoid Rust references. Obviously, calling `.as_cpp_ref()`
  works too, so this weirdness can be avoided.
* When creating some C++ data `T`, in Rust, it's annoying to have to decide a-priori
  whether it'll be Rust or C++ accessing the data. If the former, you just create
  a new `T`; if the latter you need to wrap it in `CppPin::new`. This is only
  really a problem when creating a C++ object on which you'll call methods. It
  feels like it'll be OK in practice. Possibly this can be resolved by making
  the method receiver some sort of `impl MethodReceiver<T>` generic; an implementation
  for `T` could be provided which auto-wraps it into a `CppPin` (consuming it at that
  point). This sounds messy though. A bit more thought required, but even if this
  isn't possible it doesn't sound like a really serious ergonomics problem,
  especially if we can use `#[diagnostic::on_unimplemented]` somehow to guide.

Next steps here:
* Stabilize arbitrary self types. This PR has gone far enough to show that
  there are no really serious unexpected issues there.
* Implement `ValueParam` and `RValueParam` as necessary for `CppRef` and
  `CppPin` types.
* Work on those ergonomic issues to the extent possible.
* Make a bold decision about whether autocxx should shift wholesale away from
  `&` to `CppRef<T>`. If so, this will be a significant breaking change.

Author: adetaylor

Cargo.toml

@@ -26,7 +26,7 @@ rust-version = "1.77"
 resolver = "2"
 
 [features]
-arbitrary_self_types_pointers = []
+arbitrary_self_types = []
 
 [dependencies]
 autocxx-macro = { path="macro", version="0.27.1" }

demo/src/main.rs

@@ -6,6 +6,8 @@
 // option. This file may not be copied, modified, or distributed
 // except according to those terms.
 
+#![feature(arbitrary_self_types)]
+
 use autocxx::prelude::*;
 include_cpp! {
     #include "input.h"
@@ -16,9 +18,9 @@ include_cpp! {
 
 fn main() {
     println!("Hello, world! - C++ math should say 12={}", ffi::DoMath(4));
-    let mut goat = ffi::Goat::new().within_box();
-    goat.as_mut().add_a_horn();
-    goat.as_mut().add_a_horn();
+    let goat = ffi::Goat::new().within_cpp_pin();
+    goat.add_a_horn();
+    goat.add_a_horn();
     assert_eq!(
         goat.describe().as_ref().unwrap().to_string_lossy(),
         "This goat has 2 horns."

engine/src/conversion/analysis/fun/mod.rs

@@ -1530,13 +1530,15 @@ impl<'a> FnAnalyzer<'a> {
                 let from_type = self_ty.as_ref().unwrap();
                 let from_type_path = from_type.to_type_path();
                 let to_type = to_type.to_type_path();
-                let (trait_signature, ty, method_name) = match *mutable {
+                let (trait_signature, ty, method_name): (_, Type, _) = match *mutable {
                     CastMutability::ConstToConst => (
                         parse_quote! {
-                            AsRef < #to_type >
+                            autocxx::AsCppRef < #to_type >
                         },
-                        Type::Path(from_type_path),
-                        "as_ref",
+                        parse_quote! {
+                            autocxx::CppRef< #from_type_path >
+                        },
+                        "as_cpp_ref",
                     ),
                     CastMutability::MutToConst => (
                         parse_quote! {

engine/src/conversion/codegen_rs/function_wrapper_rs.rs

@@ -172,9 +172,12 @@ impl TypeConversionPolicy {
                     _ => panic!("Not a pointer"),
                 };
                 let (ty, wrapper_name) = if is_mut {
-                    (parse_quote! { autocxx::CppMutRef<'a, #ty> }, "CppMutRef")
+                    (
+                        parse_quote! { autocxx::CppMutLtRef<'a, #ty> },
+                        "CppMutLtRef",
+                    )
                 } else {
-                    (parse_quote! { autocxx::CppRef<'a, #ty> }, "CppRef")
+                    (parse_quote! { autocxx::CppLtRef<'a, #ty> }, "CppLtRef")
                 };
                 let wrapper_name = make_ident(wrapper_name);
                 RustParamConversion::Param {
@@ -194,9 +197,9 @@ impl TypeConversionPolicy {
                     _ => panic!("Not a pointer"),
                 };
                 let ty = if is_mut {
-                    parse_quote! { &mut autocxx::CppMutRef<'a, #ty> }
+                    parse_quote! { autocxx::CppMutRef<#ty> }
                 } else {
-                    parse_quote! { &autocxx::CppRef<'a, #ty> }
+                    parse_quote! { autocxx::CppRef<#ty> }
                 };
                 RustParamConversion::Param {
                     ty,

engine/src/conversion/codegen_rs/mod.rs

@@ -17,7 +17,7 @@ pub(crate) mod unqualify;
 use indexmap::map::IndexMap as HashMap;
 use indexmap::set::IndexSet as HashSet;
 
-use autocxx_parser::{ExternCppType, IncludeCppConfig, RustFun, UnsafePolicy};
+use autocxx_parser::{ExternCppType, IncludeCppConfig, RustFun};
 
 use itertools::Itertools;
 use proc_macro2::{Span, TokenStream};
@@ -131,7 +131,6 @@ fn get_string_items() -> Vec<Item> {
 /// In practice, much of the "generation" involves connecting together
 /// existing lumps of code within the Api structures.
 pub(crate) struct RsCodeGenerator<'a> {
-    unsafe_policy: &'a UnsafePolicy,
     include_list: &'a [String],
     bindgen_mod: ItemMod,
     original_name_map: CppNameMap,
@@ -143,14 +142,12 @@ impl<'a> RsCodeGenerator<'a> {
     /// Generate code for a set of APIs that was discovered during parsing.
     pub(crate) fn generate_rs_code(
         all_apis: ApiVec<FnPhase>,
-        unsafe_policy: &'a UnsafePolicy,
         include_list: &'a [String],
         bindgen_mod: ItemMod,
         config: &'a IncludeCppConfig,
         header_name: Option<String>,
     ) -> Vec<Item> {
         let c = Self {
-            unsafe_policy,
             include_list,
             bindgen_mod,
             original_name_map: CppNameMap::new_from_apis(&all_apis),
@@ -617,11 +614,8 @@ impl<'a> RsCodeGenerator<'a> {
                 name, superclass, ..
             } => {
                 let methods = associated_methods.get(&superclass);
-                let generate_peer_constructor = subclasses_with_a_single_trivial_constructor.contains(&name.0.name) &&
-                    // TODO: Create an UnsafeCppPeerConstructor trait for calling an unsafe
-                    // constructor instead? Need to create unsafe versions of everything that uses
-                    // it too.
-                    matches!(self.unsafe_policy, UnsafePolicy::AllFunctionsSafe);
+                let generate_peer_constructor =
+                    subclasses_with_a_single_trivial_constructor.contains(&name.0.name);
                 self.generate_subclass(name, &superclass, methods, generate_peer_constructor)
             }
             Api::ExternCppType {

engine/src/conversion/conversion_tests.rs

@@ -31,7 +31,7 @@ fn do_test(input: ItemMod) {
     let inclusions = "".into();
     bc.convert(
         input,
-        UnsafePolicy::AllFunctionsSafe,
+        UnsafePolicy::ReferencesWrappedAllFunctionsSafe,
         inclusions,
         &CodegenOptions::default(),
         "",

engine/src/conversion/mod.rs

@@ -192,7 +192,6 @@ impl<'a> BridgeConverter<'a> {
                 .map_err(ConvertError::Cpp)?;
                 let rs = RsCodeGenerator::generate_rs_code(
                     analyzed_apis,
-                    &unsafe_policy,
                     self.include_list,
                     bindgen_mod,
                     self.config,

examples/reference-wrappers/src/main.rs

@@ -25,7 +25,7 @@
 
 // Necessary to be able to call methods on reference wrappers.
 // For that reason, this example only builds on nightly Rust.
-#![feature(arbitrary_self_types_pointers)]
+#![feature(arbitrary_self_types)]
 
 use autocxx::prelude::*;
 use std::pin::Pin;

integration-tests/src/lib.rs

@@ -380,6 +380,7 @@ pub fn do_run_test(
     let safety_policy = format_ident!("{}", safety_policy);
     let unexpanded_rust = quote! {
             #module_attributes
+            #![feature(arbitrary_self_types)]
 
             use autocxx::prelude::*;
 

integration-tests/tests/cpprefs_test.rs

@@ -27,7 +27,7 @@ fn run_cpprefs_test(
     generate_pods: &[&str],
 ) {
     if !arbitrary_self_types_supported() {
-        // "unsafe_references_wrapped" requires arbitrary_self_types_pointers, which requires nightly.
+        // "unsafe_references_wrapped" requires arbitrary_self_types, which requires nightly.
         return;
     }
     do_run_test(
@@ -127,7 +127,7 @@ fn test_return_reference_cpprefs() {
     let rs = quote! {
         let b = CppPin::new(ffi::Bob { a: 3, b: 4 });
         let b_ref = b.as_cpp_ref();
-        let bob = ffi::give_bob(&b_ref);
+        let bob = ffi::give_bob(b_ref);
         let val = unsafe { bob.as_ref() };
         assert_eq!(val.b, 4);
     };