New binary encoding/decoding algorithms

[gfusee]

Abstract

With the expansion of cross-chain protocols such as Axelar, Wormhole, or LayerZero, we see the emergence of cross-chain dApps. We expect the contracts of those dApps to communicate across blockchains. However, communicating means adopting a common serialization/deserialization format.

The issue

Sui's smart contracts use BCS, but this binary format is not recognized outside of Move blockchains. The reverse is also true: the Sui Move Framework doesn't provide serialization/deserialization functions for other formats.

The implementation of new binary formats in Move is possible but faces several limitations:

• sui::vector doesn't contain some functions such as reading vector slices, making it harder and less efficient to read chunks of bytes.
• Metaprogramming on structs is not possible. Move's macros are similar to Rust's declarative macros. It is impossible to perform operations on a type's fields at compile time.
• No trait system exists.

Given these limitations, the current solution to implement a new encoding format is to create helpers to encode/decode primitive Move types: u8, u64, vector<u8>, etc., and let the developer manually create encoding and decoding logic for their defined structs. This is not only time-consuming, there are security risks and optimization issues.

Solutions

There are multiple solutions that I have in mind:

• Integrating the most popular formats into the Sui Move Framework. We could have a sui::abi module working the same way as sui::bcs. This has the huge advantage of allowing serialization/deserialization to be written in the Rust runtime, but the drawback of adding niche functions to the Sui Move Framework for each binary format.
• Creating a system similar to Serde, built inside the Sui Rust runtime. This could be done by adding native Move functions in the Sui Move Framework that return a layout for a TypeName (in the case of decoding bytes) or a value (in the case of encoding), or anything that would allow the building of generic encoders/decoders in Move.
• Adding procedural macros to Move that can be attached to structs. We could imagine having annotations such as #[derive(ABIEncodable(encoder_package = ...), ABIDecodable(decoder_package = ...))]. Those macros would generate Move code using helpers functions I mentioned above. However, this may require a lot of effort to build.

I would love to hear about other ideas and solutions!

[github-actions]

Thank you for opening this issue, a team member will review it shortly. Until then, please do not interact with any users that claim to be from Sui support and do not click on any links!

[big14way]

Hi, my name is Godswill Idolor also know as Gwill. I am a blockchain engineer with 2 year of experience in backend and frontend. I am a contributor of projects like vitecare, harmony, mediano and Kindfi focusing on Smart Contracts using Rust, Frontend and Backend, and I also have experience with Cairo. StarkNet bootcamp graduate 12. I am passionate about smartcontract, which is why I enjoy designing the logic, and I have a natural ability to think critically and effectively. Along with my team, My experience in both frontend and backend enables me to carry out a wide variety of tasks to achieve set goals.

I would really love to participate in the project with this issue.
assign me

[gfusee]

Hey @big14way, to give you more context: before creating this issue I wanted to create a PR for the first solution. As explained implementing ABI directly in the Sui's Framework as a sui::abi module is a very niche feature that might not be desired by the Sui's team.

I asked Sui's team before doing anything if they are willing to accept this kind of features, or rather keep Sui's Framework more general with functions used by most of the Move developers. I got confirmation that they prefer keeping Sui's Framework more general.

I guess before implementing anything, we should debate and design it, especially given that any of the solutions I proposed might need sensitive functions to be created and thus audited.

If you have any feedback regarding the solutions, or even have another ones in mind I'd love to discuss them!

(Note: I'm not part of the Sui team)