Support dynamic payload types as request and response payload type

[elfenpiff]

Brief feature description

Currently, request response only supports static data types. It shall support dynamic payload types like publish subscribe. Therefore, we require

• support of slices as payload for requests and responses
• dynamic reallocation of servers and clients data segment
• request response example with dynamic payload
• add slice API and dynamic mem example in client.rs and server.rs

[zachary0101]

This is exactly what I am going to ask for!
I plan to use request/response message pattern to implement the client/server architecture and I need to send/receive dynamic payload. I have been waiting for this feature for a long time and I check your code everyday!
This morning my leader told me, If I can not implement our application with iceoryx2 in two weeks, we have to switch to other solution, because our project has a deadline.
I am not pushing you, but could you please share a schedule and we can arrange our plan accordingly? I'm OK with some bugs, I just need the basic function works now. Thanks!

[elfenpiff]

@zachary0101 Do you require this feature with a C++ language binding, or can you use Rust?
As an intermediate step, you could use a struct like this:

struct Temp {
    data: [u8; 1024]
}

As soon as the slice API is available, you could then switch to [u8] and in the next step to the fully dynamic version.

Our Roadmap for this feature is that we are currently working on it with the highest priority. So the next things that will be merged into iceoryx2:

1. Basic Request Response: [#429] add response channel #680
2. C/C++ language bindings for Request Response
3. v0.6 iceoryx2 release (~3rd of May, we release as soon as 2. is merged)
4. Request Response Slice API
5. Request Response Dynamic Data API
6. Cleanup of stale request response resources when a process crashes.

[zachary0101]

Thanks for your quick reply!
We need the C++ and C bindings for other application in the future, not in a hurry.
The application I am currently working on uses Rust, so I need this feature with Rust more eagerly.
I will try your suggestion tomorrow.

So, on 3rd of May you will release v0.6 with basic request/response feature but without the slice API, right?
When will the slice API be available then?
Is it possible to exchange the order of 2 and 4? ;)

[elfenpiff]

@zachary0101

The application I am currently working on uses Rust, so I need this feature with Rust more eagerly.

Here a quick path to unblock you:

In the beginning you use this implementation

// the generic parameter allows you to change the size at least at compile time
struct Data<const N: usize> {
    data: [u8; N],
}

let node = NodeBuilder::new().create::<ipc::Service>()?;

let service = node
    .service_builder(&"My/Funk/ServiceName".try_into()?)
    // substitute 1024 with whatever number you require
    .request_response::<Data<1024>, Data<1024>>()
    .open_or_create()?;

With the member Data::data you have an API that is pretty close to the slice API. If the data is insufficient, you just increase 1024 to whatever fits your use case. You might overallocate in the beginning and use more memory than required, but this is just until the dynamic memory feature has landed.

When the slice API/Dynamic Data is available, you just need to refactor the type and the loan API

let service = node
    .service_builder(&"My/Funk/ServiceName".try_into()?)
    // substitute 1024 with whatever number you require
    .request_response::<[u8], [u8]>()
    .open_or_create()?;

let server = service
    .server_builder()
   // and provide an optional hint and allocation strategy when creating a client or a server, these are just lines that need to be added
    .initial_max_slice_len(1024)
    .allocation_strategy(AllocationStrategy::PowerOfTwo)
    .create()?;

So as you see, the refactoring is minimal, and with the non-dynamic data types, you are already pretty close to what you need.

When will the slice API be available then?

I would say the latest is on the 24th of May. The heavy lifting is already done, so I suspect that we might finish dynamic memory and slices even earlier.

Is it possible to exchange the order of 2 and 4? ;)

In general, it is possible if you have a support contract. See the iceoryx2 main readme at the bottom for commercial support. In this case, we already have a sponsor who paid for that priority.

For the future, it might be interesting for you to get in touch - this is how the open-source project is financed - and it would ensure that our Roadmap aligns with your requirements. Additionally, there are also commercial extensions and tooling.

[zachary0101]

@elfenpiff Thanks for your detailed explanation. I think the workaround you provide is good enough for me at present. If we need commercial support in the future, I will discuss it with my leader. ;)

[elfenpiff]

FYI @zachary0101 as it turns out, we need the slice API to implement the C/C++ language binding. So we have to exchange the order of 2 and 4.

[zachary0101]

@elfenpiff I'm glad to hear that. Looking forward for the code. ;)