Review and update part 1 materials #10

mgeaghan · 2025-05-19T01:03:04Z

I have reviewed and updated the part 1 materials. For most of the .md files I have just updated the wording, as well as addressed Issue #3 to update section headers to use a more active voice.

I have updated both 03_hellonf.md and 04_output.md to address Issue #5 and clear up the distinction between processes and workflows. In 04_output.md I have clarified how outputs are declared but not guaranteed by the process output blocks.

I have also made significant changes to 05_inputs.md. This is mainly to address Issues #4 and #7. As part of this, I have merged the explanation of channels into this section; we can discuss whether it may be better to split off into a separate section. As part of the changes to this section, I have:

Expanded on the difference between value and queue channels.
Expanded on channel factories
Shown how to use an input channel to feed multiple input values into a process
Touched on the use of -ansi-log false for clearer terminal logs
Shown how to use an array of values to create our input channel (taken from the sequera training)
Shown how to use channel operators to manipulate the channel data
Addressed Issue Addressing publishdir as inputs #7 and mentioned that we should not use publishDir as an input to anything
Made a note about multiple input channels

I think some of this may be a bit overkill, particularly the explanation of the map() operator and the multiple input channels, so happy to discuss this and potentially remove.

…ows, declarative outputs

…nd explanation to setup

mgeaghan · 2025-05-21T23:25:22Z

docs/part1/05_inputs.md

+### Transforming each value with the `map` operator
+
+Sometimes, you will need to modify the values within a channel in a predictable way. For example, you might need to get part of a file name, or perhaps you need to take a numeric value and apply a mathematical operation to it. In these situations, you will likely want to use the `map()` operator. This takes a channel and applies a **closure** to every element in the channel.
+
+An in-depth discussion of closures is outside of the scope of this workshop, but briefly, they are blocks of code, similar to functions, that can be passed to some Nextflow operators and control how they function. With the `map()` operator, the closure defines the exact operation that should be performed on each element of the channel. A closlure is defined within curly braces like so:
+
+```groovy
+{ x ->
+    // Do something with x
+}
+```
+
+At the start of the closure definition, we declare the name of a variable that will represent each element of our channel. Here we have simply called it `x`, but it could be named anything. Next, we write an arrow operator `->`; this signifies that the value on the left (`x`) will be processed by the code on the right. Finally, we write some code to do something with our variable. For example, we can square integer values:
+
+```groovy
+{ x -> x ** 2 }
+```
+
+Or, if we have strings as inputs, we could reverse the strings:
+
+```groovy
+{ x -> x.reverse() }
+```
+
+To use the `map()` operator with a channel, we simply do:
+
+```groovy
+ch.map { x -> ... }
+```
+
+!!!question "Exercise"
+
+    Use the `map()` operator to reverse the greeting strings. 
+
+    ???Solution
+
+        ```groovy title="hello-world.nf" hl_lines="7"
+        workflow {
+
+            // Create a channel for inputs
+            greetings_array = [ 'Hello world!', 'Bonjour le monde!', 'Holà mundo' ]
+            greeting_ch = Channel.of(greetings_array)
+                .flatten()
+                .map { x -> x.reverse() }
+
+            // Emit a greeting
+            SAYHELLO(greeting_ch)
+        }
+        ```


I got a bit too into the weeds here. Probably just remove this.

mgeaghan · 2025-05-21T23:26:39Z

docs/part1/05_inputs.md

+## A note about multiple inputs
+
+The input block can be used to define multiple inputs to the process. Importantly, the number of inputs passed to the process call within the workflow must match the number of inputs defined in the process. For example:
+
+```groovy title="example.nf"
+process MYFUNCTION {
+    debug true
+
+    input:
+    val input_1
+    val input_2
+
+    output:
+    stdout
+
+    script:
+    """
+    echo $input_1 $input_2
+    """
+}
+
+workflow {
+    MYFUNCTION('Hello', 'World!')
+}
+```
+
+Another important aspect of multiple inputs is that when working with **queue channels**, they can result in **non-deterministic** results. This is because a process will execute as soon as a new value is ready for all input channels. For that reason, multiple inputs will typically be used with either value channel inputs (since they are single values that will be reused over and over again) or by using the `each` qualifier, which allows you to run the process once for every value in a collection or queue. For example:
+
+```
+process cat_message {
+    input:
+    val greeting
+    each noun
+
+    output:
+    path "message.txt"
+
+    script:
+    """
+    echo '$greeting' '$noun' > message.txt
+    """
+}
+
+workflow {
+    ch1 = Channel.of(
+        'Hello',
+        'Bonjour'
+    )
+    ch2 = Channel.of(
+        'world',
+        'everyone'
+    )
+
+    cat_message(ch1, ch2)
+}
+```
+
+This will output the following four lines (possibly in a different order):
+
+```
+Bonjour everyone
+Hello world
+Hello everyone
+Bonjour world
+```
+
+In contrast, if we didn't use the `each` qualifier with the `noun` input, we would only get two output lines, such as:
+
+```
+Hello everyone
+Bonjour world
+```
+
+Because the queues are non-deterministic, the exact combination we would get is uncertain.


I saw that multiple inputs was a question from last year's workshop, so it might be useful to keep this, but perhaps wrapped in an 'advanced material' block?

fredjaya

Minor changes, I'll update them

docs/part1/03_hellonf.md

docs/part1/04_output.md

docs/index.md

mgeaghan added 7 commits April 29, 2025 16:28

WIP: minor changes to top level pages

cca1ce9

WIP: updating part 1 wording, some more detail on processes vs workfl…

a4ec995

…ows, declarative outputs

WIP: continue updating part 1, expanding on channels and parameters

acb2859

WIP: add explanations on channel operators

8a71a91

WIP: added details on not using publishDir as input

a66e292

Finalise initial updates to part 1

e074c83

Added excalidraw image to discourage use of publishdir as input

317d2ec

mgeaghan requested a review from fredjaya May 19, 2025 01:03

mgeaghan added 2 commits May 20, 2025 14:12

Fix code highlights in part 1

739d21c

Add blank outputs to code examples in helloworld page, add code comma…

42a3fb1

…nd explanation to setup

mgeaghan commented May 21, 2025

View reviewed changes

fredjaya requested changes May 22, 2025

View reviewed changes

docs/part1/03_hellonf.md Show resolved Hide resolved

docs/part1/04_output.md Show resolved Hide resolved

docs/index.md Show resolved Hide resolved

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Review and update part 1 materials #10

Review and update part 1 materials #10

Uh oh!

mgeaghan commented May 19, 2025

Uh oh!

mgeaghan May 21, 2025

Uh oh!

mgeaghan May 21, 2025

Uh oh!

fredjaya left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Review and update part 1 materials #10

Are you sure you want to change the base?

Review and update part 1 materials #10

Uh oh!

Conversation

mgeaghan commented May 19, 2025

Uh oh!

mgeaghan May 21, 2025

Choose a reason for hiding this comment

Uh oh!

mgeaghan May 21, 2025

Choose a reason for hiding this comment

Uh oh!

fredjaya left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!