Question about snakemake-workflow-catalog, how to organize modules?

[Redmar-van-den-Berg]

In the documentation for the snakemake-workflow-catalog, all examples included use snakemake rules in separate files with "include" statements. What is the preferred way of organizing modules instead? I'm considering changing the layout of my HAMLET pipline to the recommended format, but I'm using independent modules instead of simple include statments.

[m-jahn]

Hi!
You're quite free regarding the design of your workflows. Generally I see the following design choices:

1. All rules in 1 Snakemake file -- the most basic and simple design. No includes or modules necessary, but this will not qualify as standardized usage, because the workflow catalog build process will look for a rules sub-dir which contains at least 1 .smk file. This is to support modularization.
2. 1 Snakemake file and separate .smk files, each possibly with several rules organized by topic (like input-preparation, processing, reporting, ...). This is how the new fully functional template is designed, have a look. Workflows designed like this will qualify for the catalog.
3. Multiple separate workflows can be organized as modules. Imagine you have one part of a workflow that you also want to use in other workflows, because it's so generally applicable. Then you can use modules, which are imported in your main Snakefile using the syntax:

module other_workflow:
    snakefile:
        "other_workflow/Snakefile"

use rule * from other_workflow

I would not say that this is the standard right now. This syntax is rather more useful for combining different workflows or larger modules to minimize code redundancy.

[m-jahn]

@Redmar-van-den-Berg is your question answered?

[Redmar-van-den-Berg]

If I understand correctly, only workflows that adhere to design 2 are eligible to be added to the snakemake-workflow-catalog?

[m-jahn]

Design 2 and 3 would be eligible for the standardized workflows section, that means they would get their own workflow page on the catalog. Design 1 would be listed in all other workflows without indivudual page.

The properties that are tested for each workflow and that determine if it's standardized or not are also described on the catalog docs pages.