Assignment of non-standard cache folder

[jd4ds]

Hey everybody,
we have been using this wonderful package for a long time in our team and since we moved to Posit Workbench we are wondering how to use a shared cache on the Workbench server for pins.
In the documentation of the package and in the source code I could not find any hints that the location where the caching takes place can be customized.
However, for us it would be important to be able to use a shared cache for our pins for performance reasons (cache only needs to be built once and is then available to everyone on the server) and for disk space reasons (pins duplicates in the respective user folders just take up a large amount of disk space).

Is there any possibility or is it planned (soon) to enable this or am I missing something?

Thanks and best regards,
Janis

[juliasilge]

The default behavior is for the cache to be at a location like this:

server <- pins:::rsc_server(auth = "envvar")
pins:::board_cache_path(paste0("connect-", rlang::hash(server$url)))
#> ~/Library/Caches/pins/connect-31a399613ac85efda77ffe6ff20c105a

^{Created on 2023-06-02 with reprex v2.0.2}

The only piece that goes in here is the server URL so I imagine that value would be the same for all users on a Workbench server. They are going into your own home directories, though, because users have separate caches via rappdirs::user_cache_dir().

Is there a directory or cache anywhere on your Workbench server that already is available to all your users? Let's say it is at /Library/Caches/Shared/ . Then everyone could do something like b <- board_connect(cache = "/Library/Caches/Shared/") and you would all use the same cache. As of today, everyone would need to specify that via the argument every time they create their board object, but I would be happy to fix up the environment variables that pins uses so it could be set via environment variable on the server as a whole.

What do you think about that? Does your server already have a directory like that available?

[jd4ds]

Thank you for your quick reply. Exactly, we already have a folder on Posit Workbench which we use for server-wide and cross-user caches. For instance for {renv}. So the global state of projects renv manages is no longer stored in each user's directory but shared system wide. Setting this up was very easy by explicating a unified path for the renv cache via environment variables: RENV_PATHS_CACHE (or RENV_PATHS_ROOT). https://rstudio.github.io/renv/reference/paths.html

I could imagine something like this for {pins} as well. I think only this way our code can be reproduced on different systems (e.g. also in production on Posit Connect), which would not necessarily be possible if we would hardcode the path.

Would such a customization and the addition of a specific cache directory via an environment variable be possible?

[juliasilge]

Gotcha! If you want to do this right now, you can use the env variable R_USER_CACHE_DIR (this works today with the version of pins on CRAN and will continue to work, but does change the cache location for more than just pins so you might not want to do it right now):

withr::local_envvar(list("R_USER_CACHE_DIR" = "~/Desktop"))
server <- pins:::rsc_server(auth = "envvar")
pins::board_cache_path(paste0("connect-", rlang::hash(server$url)))
#> ~/Desktop/pins/connect-31a399613ac85efda77ffe6ff20c105a

^{Created on 2023-06-02 with reprex v2.0.2}

I added some more documentation on this in #748, as well as a new env var PINS_CACHE_DIR that will work similarly (notice no "pins" added to the path):

withr::local_envvar(list("PINS_CACHE_DIR" = "~/Desktop"))
server <- pins:::rsc_server(auth = "envvar")
pins::board_cache_path(paste0("connect-", rlang::hash(server$url)))
#> ~/Desktop/connect-31a399613ac85efda77ffe6ff20c105a

^{Created on 2023-06-02 with reprex v2.0.2}

[jd4ds]

Excellent, I think that is exactly what we need!

[juliasilge]

Glad to hear this will work for your use case! You can install the new version of pins from GitHub now via pak::pak("rstudio/pins-r") (or devtools) and start using the new PINS_CACHE_DIR environment variable. There is some new documentation available, and this will go to CRAN with the next release.

Let us know if you have further questions! 🙌

[github-actions]

This issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.