Skip to content

Implement file source to integrate Galaxy with RSpace #20167

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
May 9, 2025

Conversation

kysrpex
Copy link
Contributor

@kysrpex kysrpex commented May 6, 2025

This PR implements a FilesSource that interacts with an RSpace instance and includes a file source template so that users can define their own file sources based on it.

RSpace is an electronic lab notebook (ELN) designed for documenting, managing and sharing scientific data. The central concepts in RSpace are the Workspace, the Inventory and the Gallery.

The Workspace's role is organizing and managing research activities. It allows creating documents, folders and research notebooks. The Inventory is a tool to keep track of laboratory resources, such as reagents, samples and equipment. The Gallery is where any research data represented as files is stored.

The FilesSource implemented by this module accesses the Gallery of an RSpace instance. The Gallery is organized in categories (e.g. Images, Audio, Documents, Chemistry), where each category is itself a tree structure with folders and files just like a regular filesystem, although there is a key difference: categories and their subfolders can only contain files of certain types (e.g. jpeg, bmp, png, etc. for the Images category).

This implementation is based on the rspace-client-python library, which provides a PyFilesystem2 interface to interact with RSpace's Gallery (available as GalleryFilesystem in the rspace_client.eln.fs module). It consists just of a minimal wrapper (based on the base PyFilesystem2FilesSource provided by Galaxy) around the GalleryFilesystem class.

How to test the changes?

(Select all options that apply)

  • I've included appropriate automated tests.
  • This is a refactoring of components with existing test coverage.
  • Instructions for manual testing are as follows:
    1. Run RSpace, for example using Docker Compose.
    2. Enable the file source template lib/galaxy/files/templates/examples/production_rspace.yaml on your file_source_templates.yml configuration file.
    3. Log-in with the default account "sysadmin1" and generate an API Key (under the My RSpace section).
    4. Create a new RSpace file source under Preferences -> Manage Your Remote File Sources -> Create -> RSpace, configuring the endpoint and API key.
    5. Import files from the RSpace file source (for example, one of the sample images) to your history.
    6. Export files from your history to RSpace using the "Export datasets" tool.
    7. Export your history to the RSpace file source.

License

  • I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

@kysrpex kysrpex force-pushed the rspace branch 2 times, most recently from 0b3aca6 to 7104c0c Compare May 8, 2025 08:51
@kysrpex kysrpex marked this pull request as ready for review May 8, 2025 10:31
@github-actions github-actions bot added this to the 25.0 milestone May 8, 2025
@davelopez davelopez self-requested a review May 8, 2025 22:54
kysrpex added 2 commits May 9, 2025 12:04
This commit implements a FilesSource that interacts with an RSpace [1] instance and includes a file source template so that users can define their own file sources based on it.

RSpace is an electronic lab notebook
(ELN) designed for documenting, managing and sharing scientific data. The central concepts in RSpace are the Workspace,
the Inventory and the Gallery.

The Workspace's role is organizing and managing research activities. It allows creating documents, folders and research
notebooks [2]. The Inventory is a tool to keep track of laboratory resources, such as reagents, samples and equipment
[3]. The Gallery is where any research data represented as files is stored [4].

The FilesSource implemented by this module accesses the Gallery of an RSpace instance. The Gallery is organized in
categories (e.g. Images, Audio, Documents, Chemistry), where each category is itself a tree structure with folders and
files just like a regular filesystem, although there is a key difference: categories and their subfolders can only
contain files of certain types (e.g. jpeg, bmp, png, etc. for the Images category).

This implementation is based on the rspace-client-python` library [5], which provides a PyFilesystem2 [6] interface
to interact with RSpace's Gallery (available as `GalleryFilesystem` in the rspace_client.eln.fs` module). It
consists just of a minimal wrapper (based on the base `PyFilesystem2FilesSource` provided by Galaxy) around the
`GalleryFilesystem` class.

References:

- [1] https://www.researchspace.com/
- [2] https://documentation.researchspace.com/article/bzgr8ea9e3-the-workspace
- [3] https://documentation.researchspace.com/category/zpizk20kgx-inventory
- [4] https://documentation.researchspace.com/article/sl6mo1i9do-the-gallery
- [5] https://github.com/rspace-os/rspace-client-python
- [6] https://docs.pyfilesystem.org/
There are two ways to enable file sources: globally or as file source templates. File source templates allow each user to define their own file sources, providing their credentials in the process; the newly defined file sources are only shown in the list of file sources after they have been set up. Global file sources are always shown on the list of file sources; credentials may be provided on the user preferences page.

Everything that is "personal", for example Dropbox or your own S3 buckets should be enabled using file source templates. Everything that is public AND requires no credentials (or equivalently, it is meant to be accessible to all users of the instance, as it may happen in a private lab) should also be enabled using file source templates. The remaining may be configured as global file sources.

RSpace falls in the first category, thus it makes sense to discourage configuring it as a global file source.
Copy link
Contributor

@davelopez davelopez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @kysrpex!

Looks and works fine in my local testing! 👍

@davelopez davelopez merged commit bbeb196 into galaxyproject:dev May 9, 2025
53 of 57 checks passed
@kysrpex kysrpex deleted the rspace branch May 27, 2025 13:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants