BagIt Descriptions to Libraries

[jmchilton]

I'm trying to determine if this makes sense - maybe it is too heavy though?

• https://en.wikipedia.org/wiki/BagIt
• https://www.research.manchester.ac.uk/portal/files/45989205/bagminid.pdf
• https://github.com/ini-bdds/bdbag
• https://github.com/LibraryOfCongress/bagit-python
• https://github.com/ResearchObject/bagit-ro

Would this make sense to describes groups of files for instance needed for training and allow uploads of many heterogeneous files with their metadata to libraries.

Individual manual uploads when setting up a training instance has been identified as a training pain point. We should probably either develop a YAML format Galaxy can consume to describe this or use an existing format and create entities for training-material trainings.

We could also consider these for collections and histories if this works well also.

• The following from Wikipedia makes me believe we can create bags that like out to Zenodo datasets.

  A bag can specify payload content indirectly via a "fetch.txt" file that lists URLs for content that can be fetched over the network to complete the bag.

[bgruening]

Also interesting http://www.researchobject.org. @jgoecks and @jxtx known more about this and plans about using this or similar archives to import and export data.

[jmchilton]

In response to RO from https://github.com/ResearchObject/bagit-ro:

A BagIt bag can be considered a mechanism for serialization and transport consistency, while a Research Object can be considered a way to capture identity, annotations and provenance of the resources. As such, the two formats complement each-other. They are however not directly compatible.

I think RO has an important role to play in tracking Galaxy Analyses - I'm not sure it would be the appropriate format to describe data libraries though? I'll admit there is much I don't understand though.

[jmchilton]

Possible now in dev thanks to #5220. I'll put together an issue of follow up things we can do for this work.