Skip to content

Commit 35c7f7e

Browse files
cmeesterscmeesters
committed
feat: added using wrappers section
1 parent 6e0cfad commit 35c7f7e

File tree

1 file changed

+257
-0
lines changed

1 file changed

+257
-0
lines changed

‎slides/creators/using_wrappers.tex

Lines changed: 257 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,257 @@
1+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
2+
\section{Snakemake Wrappers}
3+
{
4+
\usebackgroundtemplate{
5+
\vbox to \paperheight{\vfil\hbox to \paperwidth{\hfil\includegraphics[height=.7\paperheight]{misc/wrapper.jpg}\hfil}\vfil}
6+
}
7+
\frame{
8+
\frametitle{Using \Snakemake-Wrappers}
9+
\begin{mdframed}[tikzsetting={draw=white,fill=white,fill opacity=0.8,
10+
line width=0pt},backgroundcolor=none,leftmargin=0,
11+
rightmargin=150,innertopmargin=4pt,roundcorner=10pt]
12+
\tableofcontents[currentsection,sections={1-4},hideothersubsections]
13+
\end{mdframed}
14+
}
15+
}
16+
17+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
18+
\subsection{Using Wrappers}
19+
20+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
21+
\begin{frame}
22+
\frametitle{What is this about?}
23+
\begin{question}[Questions]
24+
\begin{itemize}
25+
\item What is a \Snakemake{} Wrapper?
26+
\item How can \Snakemake{} wrappers be used to write reproducible workflows?
27+
\end{itemize}
28+
\end{question}
29+
\begin{docs}[Objectives]
30+
\begin{enumerate}
31+
\item Introduce the concept of \Snakemake{} wrappers
32+
\item Include \Snakemake{} wrappers in your workflows
33+
\item Reproducibility of \Snakemake{} wrappers
34+
\end{enumerate}
35+
\end{docs}
36+
\end{frame}
37+
38+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
39+
\begin{frame}{Introduction to Wrappers}
40+
For a single tool, \Snakemake{} Wrappers do:
41+
\begin{itemize}[<+->]
42+
\item Document usage
43+
\item Describe the dependencies
44+
\item Provide a way to call the tool with information from Snakemake
45+
\end{itemize}
46+
This setup makes Snakemake wrappers reusable across multiple workflows.
47+
\end{frame}
48+
49+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
50+
\begin{frame}{Introduction to Wrappers}
51+
Wrappers consist of three components:
52+
\begin{enumerate}
53+
\item A description of their usage (\altverb{meta.yaml})
54+
\item A conda environment description of all dependencies of the tool (\altverb{environment.yaml})
55+
\item A python script that invokes the tool (\altverb{wrapper.py})
56+
\end{enumerate}
57+
Since they are reusable, there is a large repository online of tried and tested
58+
wrappers written by the community:
59+
\pause
60+
\begin{docs}
61+
You can browse the \lhref{https://snakemake-wrappers.readthedocs.io}{snakemake-wrapper overview page} to select your desired wrapper. New Wrappers are added frequently.
62+
\end{docs}
63+
\end{frame}
64+
65+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
66+
\begin{frame}[fragile]
67+
\frametitle{\Interlude{Dealing with multiple Extensions}}
68+
\begin{warning}
69+
So far, we were referring to \emph{one} file. But alignment software like \altverb{bwa} requires all those files with multiple extensions! {\bf This is not safe!}
70+
\end{warning}
71+
To require multiple extensions \Snakemake provides the \altverb{multiext()}-function:
72+
\begin{lstlisting}[language=Python,style=Python]
73+
rule bwa_map:
74+
input:
75+
...
76+
idx=multiext("genome", ".amb", ".fa",
77+
".ann", ".bwt", ".pac", ".sa")
78+
...
79+
\end{lstlisting}
80+
Now, \emph{all} listed extensions are required to be present for the file with the prefix \altverb{genome}!
81+
\end{frame}
82+
83+
84+
85+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
86+
\begin{frame}[fragile]{Using Wrappers - The Example}
87+
\begin{docs}
88+
The following example uses the \altverb{bwa-mem} wrapper from the \Snakemake-community wrappers repository. The example is specific and let us illustrate the usage.
89+
We will cover the usage step by step and specifically:
90+
\begin{itemize}
91+
\item How to find a wrapper?
92+
\item How to use a wrapper.
93+
\end{itemize}
94+
\end{docs}
95+
\end{frame}
96+
97+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
98+
\begin{frame}[fragile]{Using Wrappers}
99+
Using \Snakemake{} wrappers ease your life as a workflow programmer:
100+
\begin{lstlisting}[language=Python,style=Python,basicstyle=\footnotesize]
101+
rule bwa_map:
102+
input:
103+
reads=["input/{sample}_1.fq.gz",
104+
"input/{sample}_2.fq.gz"],
105+
idx=multiext("genome", ".amb",
106+
".ann", ".bwt", ".pac", ".sa"),
107+
output:
108+
"output/{sample}.bam"
109+
params:
110+
extra=r"-R '@RG\tID:{sample}\tSM:{sample}'",
111+
sorting="none",
112+
sort_order="queryname",
113+
sort_extra="",
114+
threads: 8
115+
wrapper:
116+
"v3.2.0/bio/bwa/mem"
117+
\end{lstlisting}
118+
\end{frame}
119+
120+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
121+
\begin{frame}[fragile]{Using Wrappers: Calling a Wrapper}
122+
The \altverb{wrapper} directive signals that \Snakemake{} should execute this rule with
123+
a wrapper.
124+
\begin{lstlisting}[language=Python,style=Python,basicstyle=\small]
125+
rule bwa_map:
126+
...
127+
@wrapper:@
128+
@"v3.2.0/bio/bwa/mem"@
129+
\end{lstlisting}
130+
\begin{docs}
131+
\begin{columns}
132+
\begin{column}{0.5\textwidth}
133+
Here, we instruct the code to run with the \altverb{wrapper}-directive.
134+
It specifies
135+
\begin{itemize}
136+
\item a version
137+
\item a directory
138+
\item and selects a tool.
139+
\end{itemize}
140+
\end{column}
141+
\begin{column}{0.5\textwidth}\pause
142+
The path to the tool is given relative to the workflow or a path given by the \altverb{--wrapper-prefix}
143+
flag. By default \Snakemake automatically downloads wrappers from the \lhref{https://github.com/snakemake/snakemake-wrappers}{wrappers} repository.
144+
\end{column}
145+
\end{columns}
146+
\end{docs}
147+
\end{frame}
148+
149+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
150+
\begin{frame}[fragile]{Using Wrappers: In- and Output}
151+
\altverb{input} and \altverb{output} of the rules has to match the definition of the wrapper:
152+
\begin{lstlisting}[language=Python,style=Python,basicstyle=\tiny]
153+
rule bwa_map:
154+
input:
155+
reads=["input/{sample}_1.fq.gz",
156+
"input/{sample}_2.fq.gz"],
157+
idx=multiext("genome", ".amb", ".ann",
158+
".bwt", ".pac", ".sa"),
159+
output:
160+
"output/{sample}.bam"
161+
...
162+
\end{lstlisting}
163+
\begin{docs}
164+
The \altverb{bwa-mem} wrapper expects an input object with two attributes
165+
\altverb{reads} and \altverb{idx} and writes to a single output file.\newline
166+
The example builds on short reads (hence: 2 inputs) and the \altverb{multiext}-directive allows samples to have multiple different extensions.
167+
\end{docs}
168+
\end{frame}
169+
170+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
171+
\begin{frame}[fragile]{Using Wrappers: Parameters}
172+
Parameters may be used by wrappers to influence the behaviour, settings or resource usage.
173+
\begin{lstlisting}[language=Python,style=Python,basicstyle=\tiny]
174+
rule bwa_map:
175+
...
176+
params:
177+
extra=r"-R '@RG\tID:{sample}\tSM:{sample}'",
178+
sorting="none", # Can be 'none', 'samtools' or 'picard'.
179+
sort_order="queryname", # Can be 'queryname' or 'coordinate'.
180+
sort_extra="", # Extra args for samtools/picard.
181+
threads: 8
182+
...
183+
\end{lstlisting}
184+
\begin{docs}
185+
\altverb{bwa-mem} supports custom arguments like \altverb{sort_order} and
186+
the common \altverb{extra} flag, that is used to supply command line parameters
187+
for the invocation of the tool. The \altverb{threads} and \altverb{resources}
188+
settings from a Snakemake rule are also commonly used to set memory or parallelization parameters for each tool.
189+
\end{docs}
190+
\end{frame}
191+
192+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
193+
\begin{frame}{Advantages of Wrappers}
194+
Using wrappers has many great advantages compared to \altverb{run} or \altverb{shell}:
195+
\begin{itemize}[<+->]
196+
\item Code is reusable across workflows and can be shared with others
197+
\item Complex invocations of tools do not clutter your \Snakemake{} workflow definitions
198+
\item With versioned wrappers, you workflow is easily reproducible
199+
\item Community wrappers often get you started quickly using a new tool correctly
200+
\end{itemize}
201+
\end{frame}
202+
203+
\setcounter{preframe_handson}{\value{handson}}
204+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
205+
\begin{frame}
206+
\frametitle{\HandsOn{Implement Wrapper Usage}}
207+
\setcounter{handson}{\value{preframe_handson}}
208+
Navigate to the wrapper page: \url{https://snakemake-wrappers.readthedocs.io/en/stable/} - you can search for \altverb{snakemake} + \altverb{wrappers}, too.\newline
209+
\vspace{-0.5em}
210+
\begin{question}
211+
\begin{itemize}[<+->]
212+
\item which is a suitable wrapper for our mapping rule?
213+
\item Which is a suitable wrapper for the indexing rule?
214+
\item Do you think we need to change something?
215+
\end{itemize}
216+
\end{question}
217+
\vspace{-0.5em}
218+
\pause\footnotesize
219+
\begin{docs}[Solution]
220+
\begin{itemize}[<+->]
221+
\item \altverb{v5.7.0/bio/bwa/mem}
222+
\item \altverb{v5.7.0/bio/samtools/index}
223+
\item the suffixes for our reference do not match
224+
\item we need to select \altverb{sorting="samtools} - why?
225+
\item we need to select \altverb{sort_order="coordinate"} - why?
226+
\end{itemize}
227+
\end{docs}
228+
\end{frame}
229+
230+
231+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
232+
\begin{frame}
233+
\frametitle{\HandsOn{Implement Wrapper Usage II}}
234+
\setcounter{handson}{\value{preframe_handson}}
235+
Replace the rules \altverb{bwa_map} and \altverb{samtools_index} with wrapper rules.
236+
\begin{hint}
237+
Remember
238+
\begin{itemize}
239+
\item Take the example code from the wrappers web page.
240+
\item for bwa and sorting we use: \altverb{v5.7.0/bio/bwa/mem}
241+
\item for the indexing we use \altverb{v5.7.0/bio/samtools/index}
242+
\item the suffixes for our reference do not match, enter the appropriate ones
243+
\item we need to select \altverb{sorting="samtools} and
244+
\item \altverb{sort_order="coordinate"} for the wrapper based \altverb{bwa_map} rule.
245+
\end{itemize}
246+
\end{hint}
247+
248+
\begin{task}
249+
Try it!
250+
\end{task}
251+
\end{frame}
252+
253+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
254+
\begin{frame}
255+
\frametitle{Solution to the Wrapper Exercise}
256+
A solution is in \altverb{<++course.pathtosolutions++>/13_Snakefile_wrapper}.
257+
\end{frame}

0 commit comments

Comments
 (0)