-
Notifications
You must be signed in to change notification settings - Fork 2
/
Copy pathREADME.Rmd
123 lines (88 loc) · 4.27 KB
/
README.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
---
output: github_document
---
<!-- README.md is generated from README.Rmd. Please edit that file -->
```{r, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.path = "man/figures/README-",
out.width = "100%"
)
```
# sakura
<!-- badges: start -->
[](https://lifecycle.r-lib.org/articles/stages.html#experimental)
[](https://CRAN.R-project.org/package=sakura)
[](https://github.com/shikokuchuo/sakura/actions)
[](https://app.codecov.io/gh/shikokuchuo/sakura)
<!-- badges: end -->
```
________
/\ sa \
/ \ ku \
\ / ra /
\/_______/
```
### Extension to R Serialization
Extends the functionality of R serialization by augmenting the built-in reference hook system. This enhanced implementation allows an integrated single-pass operation that combines R serialization with third-party serialization methods.
Facilitates the serialization of even complex R objects, which contain non-system reference objects, such as those accessed via external pointers, to enable their use in parallel and distributed computing.
This package was a request from a meeting of the [R Consortium](https://r-consortium.org/) [Marshalling and Serialization Working Group](https://github.com/RConsortium/marshalling-wg/) held at useR!2024 in Salzburg, Austria. It is designed to eventually provide a common framework for marshalling in R.
It extracts the functionality embedded within the [mirai](https://github.com/r-lib/mirai) async framework for use in other contexts.
### Installation
Install the current release from CRAN:
```{r cran, eval=FALSE}
install.packages("sakura")
```
Or the development version using:
```{r devinstall, eval=FALSE}
pak::pak("shikokuchuo/sakura")
```
### Overview
Some R objects by their nature cannot be serialized, such as those accessed via an external pointer.
Using the [`arrow`](https://arrow.apache.org/docs/r/) package as an example:
```{r arrowfail,error=TRUE}
library(arrow, warn.conflicts = FALSE)
obj <- list(as_arrow_table(iris), as_arrow_table(mtcars))
unserialize(serialize(obj, NULL))
```
In such cases, `sakura::serial_config()` can be used to create custom serialization configurations, specifying functions that hook into R's native serialization mechanism for reference objects ('refhooks').
```{r arrowcfg}
cfg <- sakura::serial_config(
"ArrowTabular",
arrow::write_to_raw,
function(x) arrow::read_ipc_stream(x, as_data_frame = FALSE)
)
```
This configuration can then be supplied as the 'hook' argument for `sakura::serialize()` and `sakura::unserialize()`.
```{r arrowpass}
sakura::unserialize(sakura::serialize(obj, cfg), cfg)
```
This time, the arrow tables are handled seamlessly.
Using `torch` as another example:
```{r torchfail, error=TRUE}
library(torch)
x <- list(torch_rand(5L), runif(5L))
unserialize(serialize(x, NULL))
```
Base R serialization above fails, but `sakura` serialization succeeds:
```{r torchpass}
cfg <- sakura::serial_config("torch_tensor", torch::torch_serialize, torch::torch_load)
sakura::unserialize(sakura::serialize(x, cfg), cfg)
```
### C Interface
A low-level interface is provided for use by other packages. The following C callables are registered:
```c
sakura_serialize_init;
sakura_unserialize_init;
sakura_serialize;
sakura_unserialize;
```
Their function signatures may be inspected in `src/sakura.h`.
### Acknowledgements
We would like to thank in particular:
- [R Core](https://www.r-project.org/contributors.html) for providing the interface to the R serialization mechanism.
- [Luke Tierney](https://github.com/ltierney/) and [Mike Cheng](https://github.com/coolbutuseless) for their meticulous efforts in documenting the serialization interface.
- [Daniel Falbel](https://github.com/dfalbel) for discussion around an efficient solution to serialization and transmission of torch tensors.
--
Please note that this project is released with a [Contributor Code of Conduct](https://shikokuchuo.net/sakura/CODE_OF_CONDUCT.html). By participating in this project you agree to abide by its terms.