Skip to content

Commit 6307aa0

Browse files
bendkbendk
committed
Adding ADR-0009
This revisits the decision in `ADR-0005` and explores using going back to handle maps to pass objects across the FFI.
1 parent 9c353c5 commit 6307aa0

File tree

3 files changed

+299
-1
lines changed

3 files changed

+299
-1
lines changed

‎docs/adr/0005-arc-pointers.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Use raw `Arc<T>` pointers to pass objects across the FFI
22

3-
* Status: proposed
3+
* Status: accepted
44
* Deciders: mhammond, rfkelly
55
* Consulted: travis, jhugman, dmose
66
* Date: 2021-04-19

‎docs/adr/0009-handles.md

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
# Use handle map handles to pass objects across the FFI
2+
3+
* Status: proposed
4+
* Deciders:
5+
* Consulted:
6+
7+
Discussion and approval:
8+
9+
ADR-0005 discussion: [PR 430](https://github.com/mozilla/uniffi-rs/pull/430).
10+
11+
## Context and Problem Statement
12+
13+
UniFFI currently passes objects from Rust to the foreign side by leaking an Arc reference into a word-sized opaque pointer and passing it across the FFI.
14+
The basic approach uses `Arc::into_raw` / `Arc::from_raw` and was chosen in [ADR-0005](./0005-arc-pointers.md) for several reasons:
15+
16+
1. Clearer generated code.
17+
2. Ability to pass objects as arguments (https://github.com/mozilla/uniffi-rs/issues/40).
18+
This was deemed difficult to do with the existing codegen + HandleMap.
19+
3. Ability to track object identity (https://github.com/mozilla/uniffi-rs/issues/197). If two function calls return the same object, then this should result in an identical object on the foreign side.
20+
4. Increased performance.
21+
22+
Recently, this approach was extended to work with unsized types (`Arc<dyn Trait>`), which are normally wide pointers (i.e double-word sized).
23+
For these types, we box the Arc to create `Box<Arc<dyn Trait>>`, then leak the box pointer.
24+
This results in a regular, word-sized, pointer since `Arc<dyn Trait>` is sized (2 words) even when `dyn Trait` is not.
25+
26+
Now that we have several years of experience, it's a good time to revisit some of the reasoning in ADR-0005 because it seems like we're not getting the benefits we wanted:
27+
28+
* The code that deals with these isn't so clear, especially when we have to deal with unsized types (for example
29+
the RustFuture
30+
[allocation](https://github.com/mozilla/uniffi-rs/blob/fbc6631953a889c7af6e5f1af94de9242589b75b/uniffi_core/src/ffi/rustfuture/mod.rs#L56-L63) / [dellocation](https://github.com/mozilla/uniffi-rs/blob/fbc6631953a889c7af6e5f1af94de9242589b75b/uniffi_core/src/ffi/rustfuture/mod.rs#L124-L125) or the similar code for trait interfaces).
31+
* The codegen has progressed and it would be easy to support `[2]`.
32+
We could simply `clone` the handle as part of the `lower()` call.
33+
* We've never implemented the reverse identity map needed for `[3]`.
34+
The `NimbusClient` example given in https://github.com/mozilla/uniffi-rs/issues/419 would still fail today.
35+
Given that there has been little to no demand for this feature, this should be changed to a non-goal.
36+
* The performance benefit decreases when discussing unsized types which require an additional layer of boxing.
37+
In that case, instead of a strict decrease in work, we are trading a `HandleMap` insertion for a Box allocation.
38+
This is a complex tradeoff, with the box allocation likely being faster, but not by much.
39+
40+
Furthermore, practice has shown that dealing with raw pointers makes debugging difficult, with errors often resulting in segfaults or UB.
41+
Dealing with any sort of FFI handle is going to be error prone, but at least with a handle map we can generate better error messages and correct stack traces.
42+
There are also more error modes with this code.
43+
44+
### Safety
45+
46+
ADR-0005 says "We believe the additional safety offered by `HandleMap`s is far less important for this use-case, because the code using these pointers is generated instead of hand-written."
47+
48+
While it's certainly true safety benefits matter less for generated code, it's also true that UniFFI is much more complex now then when ADR-0005 was decided.
49+
We have introduced callback interfaces, trait interfaces, Future handles for async functions, etc.
50+
All of these introduce additional failure cases, for example #1797, which means that relatively small safety benefits are more valuable.
51+
52+
### Foreign handles
53+
54+
A related question is how to handle handles to foreign objects that are passed into Rust.
55+
However, that question is orthogonal to this one and is out-of-scope for this ADR.
56+
57+
## Considered Options
58+
59+
### [Option 1] Continue using raw Arc pointers to pass Rust objects across the FFI
60+
61+
Stay with the current status quo.
62+
63+
### [Option 2] Use the old `HandleMap` to pass Rust objects across the FFI
64+
65+
We could switch back to the old handle map code, which is still around in the [ffi-support crate](https://github.com/mozilla/ffi-support/blob/main/src/handle_map.rs).
66+
This implements a relatively simple handle-map that uses a `RWLock` to manage concurrency.
67+
68+
See [../handles.md] for details on how this would work.
69+
70+
Handles are passed as a `u64` values, but they only actually use 48 bits.
71+
This works better with JS, where the `Value` type only supports integers up to 53-bits wide.
72+
73+
### [Option 3] Use a `HandleMap` with more performant/complex concurrency strategy
74+
75+
We could switch to something like the [handle map implementation from #1808](https://github.com/bendk/uniffi-rs/blob/d305f7e47203b260e2e44009e37e7435fd554eaa/uniffi_core/src/ffi/slab.rs).
76+
The struct in that code was named `Slab` because it was inspired by the `tokio` `slab` crate.
77+
However, it's very similar to the original UniFFI `HandleMap` and this PR will call it a `HandleMap` to follow in that tradition.
78+
79+
See [../handles.md] for details on how this would work.
80+
81+
### [Option 4] Use a 3rd-party crate to pass Rust objects across the FFI
82+
83+
We could also use a 3rd-party crate to handle this.
84+
The `sharded-slab` crate promises lock-free concurrency and supports generation counters.
85+
86+
## Decision Drivers
87+
88+
## Decision Outcome
89+
90+
???
91+
92+
## Pros and Cons of the Options
93+
94+
### [Option 1] Continue using raw Arc pointers to pass Rust objects across the FFI
95+
96+
* Good, because it has the fastest performance, especially for sized types.
97+
* Good, because it doesn't require code changes.
98+
* Bad, because it's hard to debug errors.
99+
100+
### [Option 2] Use the original handle map to pass Rust objects across the FFI
101+
102+
* Good, because it's easier to debug errors.
103+
* Bad, because it requires a read-write lock.
104+
In particular, it seems bad that `insert`/`remove` can block `get`.
105+
* Good, because it works better with Javascript
106+
* Good, because it works with any type, not just `Arc<T>`.
107+
For example, we might want to pass a handle to a [oneshot::Sender](https://docs.rs/oneshot/latest/oneshot/) across the FFI to implement async callback interface methods.
108+
109+
### [Option 3] Use a handle map with a simpler concurrency strategy
110+
111+
* Good, because it's easier to debug errors.
112+
* Good because `get` doesn't require a lock.
113+
* Bad because `insert` and `remove` requires a lock.
114+
* Bad, because it requires consumers to depend on `append-only-vec`.
115+
However, this is a quite small crate.
116+
* Good, because it works better with Javascript
117+
* Good, because it works with any type, not just `Arc<T>`.
118+
119+
### [Option 4] Use a 3rd-party crate to pass Rust objects across the FFI
120+
121+
* Good, because it's easier to debug errors.
122+
* Bad, because it requires consumers to take this dependency.
123+
* Bad, because it makes it harder to implement custom functionality.
124+
For example, supporting clone to fix https://github.com/mozilla/uniffi-rs/issues/1797 or adding a foreign bit to improve trait interface handling.

‎docs/handles.md

Lines changed: 174 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,174 @@
1+
# How do UniFFI handles and handle maps work?
2+
3+
UniFFI uses handles to pass Rust objects across the FFI to the foreign code.
4+
The handles point to an entry inside a `HandleMap`
5+
6+
## HandleMap
7+
8+
A `HandleMap` is a `Vec` where each item is either:
9+
10+
- **Occupied**
11+
- The foreign side holds a handle that's associated with the entry.
12+
- Stores a `T` value (typically an `Arc<>`).
13+
- Stores a `generation` counter used to detect use-after-free bugs
14+
- Stores a `map-id` value used to detect handles used with the wrong `HandleMap`
15+
- **Vacant**
16+
- Each vacant entry stores the index of the next vacant entry.
17+
These form a linked-list of available entries and allow us to quickly allocate a new entry in the `HandleMap`
18+
- Stores the `generation` counter value from the last time it was occupied, or 0 if it was never occupied.
19+
20+
Furthermore, each `HandleMap` stores its own `next` value, which points to the first vacant entry on the free list.
21+
The value u32::MAX indicates indicates there is no next item and is represented by the `EOL` const.
22+
23+
Here's an example `HandleMap`:
24+
25+
```
26+
----------------------------------------------------------------
27+
| OCCUPIED | VACANT | OCCUPIED |
28+
| item: Foo | next: EOL | item: Bar |
29+
| generation: 100 | generation: 40 | generation: 30 |
30+
----------------------------------------------------------------
31+
32+
HandleMap.next: 1
33+
```
34+
35+
### Inserting entries
36+
37+
To insert a new entry:
38+
- Remove the first entry in the free list,
39+
- Convert it to an `OCCUPIED`
40+
- Increment the generation counter
41+
42+
For example inserting a `Baz` entry in the above `HandleMap` would result in:
43+
44+
```
45+
----------------------------------------------------------------
46+
| OCCUPIED | OCCUPIED | OCCUPIED |
47+
| item: Foo | item: Baz | item: Bar |
48+
| generation: 100 | generation: 41 | generation: 30 |
49+
----------------------------------------------------------------
50+
51+
HandleMap.next: EOL
52+
```
53+
54+
If there are no vacant entries, then we append an entry to the end of the list.
55+
For example, inserting a `Qux` entry in the above `HandleMap` would result in:
56+
57+
```
58+
-------------------------------------------------------------------------------------
59+
| OCCUPIED | OCCUPIED | OCCUPIED | OCCUPIED |
60+
| item: Foo | item: Baz | item: Bar | item: Qux |
61+
| generation: 100 | generation: 41 | generation: 30 | generation: 0 |
62+
-------------------------------------------------------------------------------------
63+
64+
HandleMap.next: EOL
65+
```
66+
67+
### Removing entries
68+
69+
To remove an entry:
70+
- Convert it to `VACANT`
71+
- Add it to the head of the free list.
72+
73+
For example, removing the `Foo` entry from the above handle map would result in:
74+
75+
```
76+
-------------------------------------------------------------------------------------
77+
| VACANT | OCCUPIED | OCCUPIED | OCCUPIED |
78+
| next: EOL | item: Baz | item: Bar | item: Qux |
79+
| generation: 100 | generation: 41 | generation: 30 | generation: 0 |
80+
-------------------------------------------------------------------------------------
81+
82+
HandleMap.next: 0
83+
```
84+
85+
Removing the `Bar` entry after that would result in:
86+
87+
```
88+
-------------------------------------------------------------------------------------
89+
| VACANT | OCCUPIED | OCCUPIED | OCCUPIED |
90+
| next: EOL | item: Baz | next: 0 | item: Qux |
91+
| generation: 100 | generation: 41 | generation: 30 | generation: 0 |
92+
-------------------------------------------------------------------------------------
93+
94+
HandleMap.next: 2
95+
```
96+
97+
### Getting entries
98+
99+
When an entry is inserted, we return a `Handle`.
100+
This is a 64-bit integer, segmented as follows:
101+
- Bits 0-32: `Vec` index
102+
- Bit 32: foreign bit that's set for handles for foreign objects, but not Rust objects.
103+
This allows us to differentiate trait interface implementations.
104+
- Bits 33-40: map id -- a unique value that corresponds to the map that generated the handle
105+
- Bits 40-48: generation counter
106+
- Bits 48-64: unused
107+
108+
When the foreign code passes the Rust code a handle, we use it to get the entry as follows:
109+
110+
- Use the index to get the entry in the raw `Vec`
111+
- Check that the entry is `OCCUPIED`, the generation counter value matches, and the map_id matches.
112+
- Get the stored item and do something with it. Usually this means cloning the `Arc<>`.
113+
114+
These checks can usually ensure that handles are only used with the `HandleMap` that generated them and that they aren't used after the entry is removed.
115+
However, this is limited by the bit-width of the handle segments:
116+
117+
- Because the generation counter is 8-bits, we will fail to detect use-after-free bugs if an entry has been reused exactly 256 items or some multiple of 256.
118+
- Because the map id is only 7-bits, we may fail to detect handles being used with the wrong map if we generate over 128 `HandleMap` tables.
119+
This can only happen if there more than 100 user-defined types and less than 1% of the time in that case.
120+
121+
### Handle map creation / management
122+
123+
The Rust codegen creates a static `HandleMap` for each object type that needs to be sent across the FFI, for example:
124+
- `HandleMap<Arc<T>>` for each object type exposed by UniFFI.
125+
- `HandleMap<Arc<dyn Trait>>` for each trait interface exposed by UniFFI.
126+
- `HandleMap<Arc<dyn RustFutureFfi<FfiType>>>` for each FFI type. This is used to implement async Rust functions.
127+
- `HandleMap<oneshot::Sender<FfiType>>` for each FFI type. This will be used to implement async callback methods.
128+
129+
The `HandleAlloc` trait manages access to the static `HandleMap` instances and provides the following methods:
130+
- `insert(value: Self) -> Handle` insert a new entry into the handle map
131+
- `remove(handle: Handle) -> Self` remove an entry from the handle map
132+
- `get(handle: Handle) -> Self` get a cloned object from the handle map without removing the entry.
133+
- `clone_handle(handle: Handle) -> Handle` get a cloned handle that refers to the same object.
134+
135+
If the user defines a type `Foo` in their interface then:
136+
- `<Arc<Foo> as HandleAlloc>::insert` is called when lowering `Foo` to pass it across the FFI to the foreign side.
137+
- `<Arc<Foo> as HandleAlloc>::get` is called when lifting `Foo` after it's passed back across the FFI to Rust.
138+
- `<Arc<Foo> as HandleAlloc>::clone_handle` is called when the foreign side needs to clone the handle.
139+
See https://github.com/mozilla/uniffi-rs/issues/1797 for why this is needed.
140+
- `<Arc<Foo> as HandleAlloc>::remove` is called when the foreign side calls the `free` function for the object.
141+
142+
Extra details:
143+
- The trait is actually `HandleAlloc<UT>`, where `UT` is the "UniFFI Tag" type. See `uniffi_core/src/ffi_converter_traits.rs` for details.
144+
- The last two `HandleAlloc` methods are only implemented for `T: Clone`, which is true for the `Arc<>` cases, but not `oneshot::Sender`.
145+
This is fine because we only use `insert`/`remove` for the `oneshot::Sender` case.
146+
147+
### Concurrency
148+
149+
`insert` and `remove` require serialization since there's no way to atomically update the free list.
150+
In general, `get` does not require any serialization since it will only read occupied entries, while `insert` and `remove` only access vacant entries.
151+
However, there are 2 edge cases where `get` does access the same entries as `insert`/`remove`:
152+
153+
- If `insert` causes the Vec to grow this may cause the entire array to be moved, which will affect `get`
154+
- If the foreign code has a use-after-free bug, then `get` may access the same entry as an `insert`/`remove` operation.
155+
156+
UniFFI uses the following system to handle this:
157+
158+
- A standard `Mutex` is used to serialize `insert` and `remove`.
159+
- We use the `append_only_vec` crate, which avoids moving the array when the `Vec` grows.
160+
- Each entry has a 8-bit read-write spin-lock to avoid issues in the face of use-after-free bugs.
161+
This lock will only be contested if there's a use-after-free bug.
162+
163+
### Concurrency: alternative option if we choose 2 from the ADR. This one is simpler, but slower.
164+
165+
To allow concurrent access, a `RwLock` is used to protect the entire `HandleMap`.
166+
`insert` and `remove` acquire the write lock while accessing entries acquires the read lock.
167+
168+
### Space usage
169+
170+
The `HandleMap` adds an extra 64-bits of memory to each occupied item, which is the lower-limit on a 64-bit machine.
171+
This means that `HandleMap` tables that store normal `Arc` pointers add ~100% extra space overhead and ones that store wide-pointers add ~50% overhead.
172+
`HandleMap` tables don't have any way of reclaiming unused space after items are removed.
173+
174+
This is can be a very large amount of memory, but in practice libraries only generate a relatively small amount of handles.

0 commit comments

Comments
 (0)