%\VignetteIndexEntry{Sequenza user guide}
%\VignetteDepends{sequenza}
%\VignetteKeywords{LOH}
%\VignetteKeywords{Copy number}
%\VignetteKeywords{Cancer sequencing}
%\VignettePackage{sequenza}

\documentclass[10pt,a4paper]{article}
%\usepackage{a4wide}
%\usepackage{geometry}
\usepackage{hyperref}
%\usepackage{subfigure}
\usepackage{listings}
\usepackage{float}
\lstdefinestyle{BashInputStyle}{
  language=bash,
  basicstyle=\small\sffamily,
  numbers=left,
  numberstyle=\tiny,
  numbersep=3pt,
  columns=fullflexible,
  linewidth=0.9\linewidth,
  xleftmargin=0.1\linewidth
}

\newcommand{\shellcmd}[1]{\\\indent\indent\texttt{\footnotesize\# #1}\\}
\newcommand{\Rfunction}[1]{{\texttt{#1}}}
\newcommand{\Robject}[1]{{\texttt{#1}}}
\newcommand{\Rpackage}[1]{{\textit{#1}}}

\author{Francesco Favero\footnote{favero@cbs.dtu.dk}, Tejal Joshi, Andrea M. Marquard, Aron C. Eklund}

\begin{document}
\SweaveOpts{concordance=TRUE}
\title{sequenza usage example}
\maketitle

\tableofcontents

\section{Abstract}
Deep sequence of tumor DNA along with corresponding normal DNA can provide a valuable perspective on the mutations and aberrations that characterize the tumor. However, analysis of this data can be impeded by tumor cellularity and heterogeneity and by unwieldy data. Here we describe \textit{Sequenza}, which comprises a fast python-based pre-processor and an R-based analysis package. \textit{Sequenza} enables the efficient estimation of tumor cellularity and ploidy, and generation of copy number, loss-of-heterozygosity, and mutation frequency profiles. 

This document details a typical analysis of matched tumor-normal exome sequence data using \textit{sequenza}.

\section{Getting started}

\subsection{Minimum requirements}
\begin{itemize}
\item Software: R, Python
\item Operating system: Linux, OS X, Windows
\item Memory: Minimum 4 GB of RAM.  Recommended >8 GB.
\item Disk space: 1.5 GB for sample 
\item R version: 2.15.1 
\item Python version: 2.7 (or Pypy 2.*) 
\end{itemize}

\subsection{Installation}
In order to install sequenza, you can download the package from the nearest CRAN mirror doing:

<<echo=FALSE>>=
  options(width = 60, strict.width = "wrap")
@

<<label=instLib, echo=TRUE, eval=FALSE>>=
install.packages("sequenza")
@

\subsection{Note on the helper program \texttt{sequenza-utils.py}.}
For convenience and efficiency we have implemented pre-processing algorithms in a standalone (not called from R) Python program. This program is provided with the R package; its exact location can be found like this:

<<label=findexec, echo=TRUE, eval=FALSE>>=
system.file("exec", "sequenza-utils.py", package="sequenza")
@
You may wish to copy this program to a location on your path.


\subsection{Workflow overview}
A typical workflow developed with Sequenza on pre-aligned sequencing files (BAM format) is structured as follows:
\begin{enumerate}
\item Convert pileup to \textit{seqz} format 
\item Normalization of depth ratio
\item Allele-specific segmentation using the depth ratio and the B allele frequencies (BAFs)
\item Infer cellularity and ploidy by model fitting
\item Call CNV and variant alleles
\end{enumerate}



\section{Preparing inputs for Sequenza}

In order to obtain precise mutational and aberration patterns in a tumor sample, Sequenza requires a matched normal sample from the same patient. Typically, the following files are needed to get started with Sequenza.
\begin{enumerate}
\item A BAM file (or a derived pileup file) from the tumor specimen.
\item A BAM file (or a derived pileup file) from the normal specimen.
\item A FASTA reference genomic sequence file (to extract GC-content information, and to transform BAM to pileup if needed.)
\end{enumerate}

Alternatively, it is possible to use the output of VarScan2\cite{Koboldt2012} (\url{http://varscan.sourceforge.net}), which requires a similar approach and the generation of pileups as well. In this case, you can skip ahead to the ``Converting VarScan2 output to seqz'' section below.

The genome sequence file can be obtained from e.g. \url{http://hgdownload.cse.ucsc.edu/downloads.html} or \url{http://www.ensembl.org/info/data/ftp/index.html}.


\subsection{Generating pileup files from BAM files}

We recommend using pre-processed and quality-filtered BAM files to obtain pileup calls for both samples. 

Pileup files can be generated using \texttt{samtools}\cite{Li2009}. For example:

\begin{lstlisting}[style=BashInputStyle]
   samtools mpileup -f hg19.fasta -Q 20 normal.bam | gzip > normal.pileup.gz
   samtools mpileup -f hg19.fasta -Q 20 tumor.bam  | gzip >  tumor.pileup.gz
\end{lstlisting}



\subsection{Generating a genome-wide GC content file}

The genome-wide GC content is used to normalize the depth ratio. To obtain the GC content file, you can use a function from \texttt{sequenza-utils.py} to extract the average GC content from a genome FASTA file using fixed genomic windows.

The following example calculates GC content in 50-base windows:
\begin{lstlisting}[style=BashInputStyle]
   sequenza-utils.py GC-windows -w 50 hg19.fa | gzip > hg19.gc50Base.txt.gz
\end{lstlisting}

Alternatively, you can download the gc5Base file from golden path (\url{http://hgdownload-test.cse.ucsc.edu/goldenPath/hg19/gc5Base/}).


\subsection{Generate a seqz file}

A \textit{seqz} file contains genotype information, alleles and mutation frequency, and  other features. This file is used as input for the R-based part of Sequenza.

The seqz file is generated like this:
\begin{lstlisting}[style=BashInputStyle]
   sequenza-utils.py pileup2seqz -gc hg19.gc50Base.txt.gz \
                                   -n normal.pileup.gz \
                                   -t tumor.pileup.gz | gzip > out.seqz.gz
\end{lstlisting}

Alternatively, if you haven't already generated the pileup files, and you are not interested in storing the pileup for further use, you can use the function \texttt{bam2seqz} which generates two temporary named pipes (\textit{FIFOs}) converting on the fly to pileup using \texttt{samtools} without storing the pileup file:
\begin{lstlisting}[style=BashInputStyle]
   sequenza-utils.py bam2seqz -gc hg19.gc50Base.txt.gz 
                              --fasta hg19.fasta \
                              -n normal.fifo \
                              -t tumor.fifo | gzip > out.seqz.gz
\end{lstlisting}

To reduce the size of the seqz file, we recommend the use of a binning function provided in \texttt{sequenza-utils.py}.
This binning decreases the memory requirement to load the data into R, and it also speeds up the processing of the sample:

\begin{lstlisting}[style=BashInputStyle]
   sequenza-utils.py seqz-binning -w  50 \
                                  -s out.seqz.gz | gzip > out_small.seqz.gz
\end{lstlisting}

Where the parameter \textit{-w} indicate a window size (in bases), to be used for the binning. This binning has only a negligible effect on Sequenza output. The heterozygous positions and the positions carrying variant calls are not affected by binning. 

\subsection{Converting VarScan2 output to seqz}

Since many projects might already have been processed with VarScan2, it can be convenient to be able to import such results.
For this purpose a simple function is provided within the package, to convert the output of the \textit{somatic} and \textit{copynumber} programs of the VarScan2 suite into the \textit{seqz} format.

<<label=varscan, echo=TRUE, eval=FALSE>>=
snp <- read.table("varscan.snp", header = TRUE, sep = "\t")
cnv <- read.table("varscan.copynumber", header = TRUE, sep = "\t")
seqz.data <- VarScan2seqz(varscan.somatic = snp, varscan.copynumber = cnv)

write.table(seqz.data, "my.sample.seqz", col.names = TRUE, row.names = FALSE, sep = "\t")
@

For \textit{whole genome} sequencing, the information in the \textit{varscan.snp} could be sufficient to estimate the ploidy and cellularity, and define the copy number profile and mutations; hence the \texttt{varscan.copynumber} argument is optional. However, for \textit{exome} sequencing, we recommend that you supply \texttt{varscan.copynumber}.

\section{Exploring the \textit{seqz} file and depth ratio normalization details}

After the aligned sequence data have been pre-processed, the \texttt{sequenza} R package handles all the normalization and analysis steps. Thus, the remainder of this vignette will take place in R.

<<label=loadLib, echo=TRUE>>=
library("sequenza")
@

\subsection{Read the \textit{seqz} file}
In the package we provide an example file. To find the complete path of the example data file:
<<label=setFile, echo=TRUE, eval=FALSE>>=
data.file <-  system.file("data", "example.seqz.txt.gz", package = "sequenza")
data.file
@

<<label=setFile2, echo=FALSE, eval=TRUE>>=
data.file <-  system.file("data", "example.seqz.txt.gz", package = "sequenza")
@
The seqz file can be read all at once, but processing one chromosome at a time is less demanding on computational resources, especially while processing whole genome data, and might be preferable in case of limited computational resources.

Read only the data corresponding to chromosome 1:
<<label=readAfreqChr, echo=TRUE, eval=FALSE>>=
seqz.data <- read.seqz(data.file, chr.name = "1")
@

Alternatively, read all data at once:
<<label=readAfreq, echo=TRUE>>=
seqz.data <- read.seqz(data.file)
@

<<>>==
str(seqz.data, vec.len = 2)
@

The files can be read even faster; after mapping the chromosomes location in the file, it is possible to select the specific lines of the file to read. See the man page of \texttt{read.seqz} for an example.

\subsection{Quality control}

Each aligned base, in the next generation sequencing, is associated with a quality score. The \texttt{sequenza-utils.py} software is capable of filtering out bases with a quality score lower then a specified value (default, 20). The number of reads that have passed the filter is returned in the column \textit{good.reads}, while the \textit{depth.tumor} column contains the raw depth indicated in the pileup.

\subsection{Normalization of depth ratio}

The GC content bias affects most of the samples; however, some samples are more biased than others. We attempt to remove this bias by normalizing with the mean depth ratio value of a corresponding GC content value.

For efficiency, the function \texttt{gc.sample.stats} simultaneously gathers GC-content information from the entire file and in the meantime maps the chromosome position in the file (thus enabling direct access to a specified chromosome later, see \texttt{?read.seqz}):

<<label=depthRGCnormall, echo=TRUE, results=hide>>=
gc.stats <- gc.sample.stats(data.file)
@

<<label=gcstr, echo=TRUE >>=
str(gc.stats)
@

Or alternatively, it is possible to collect the GC-contents information from an object already loaded in the environment.
<<label=depthRGCnorm, echo=TRUE, eval=FALSE>>=
gc.stats <- gc.norm(x = seqz.data$depth.ratio,
                    gc = seqz.data$GC.percent)
@

In either case the the normalization of the depth ratio is performed in the following way:
<<label=useGCmedians, echo=TRUE>>=
gc.vect  <- setNames(gc.stats$raw.mean, gc.stats$gc.values)

seqz.data$adjusted.ratio <- seqz.data$depth.ratio / 
                           gc.vect[as.character(seqz.data$GC.percent)]                       
@

\begin{figure}[H]
      \centering
<<label=GCfig, fig = TRUE, width = 8, height = 4.5>>==
par(mfrow = c(1,2), cex = 1, las = 1, bty = 'l')
matplot(gc.stats$gc.values, gc.stats$raw,
        type = 'b', col = 1, pch = c(1, 19, 1), lty = c(2, 1, 2),
        xlab = 'GC content (%)', ylab = 'Uncorrected depth ratio')
legend('topright', legend = colnames(gc.stats$raw), pch = c(1, 19, 1))
hist2(seqz.data$depth.ratio, seqz.data$adjusted.ratio,
      breaks = prettyLog, key = vkey, panel.first = abline(0, 1, lty = 2),
      xlab = 'Uncorrected depth ratio', ylab = 'GC-adjusted depth ratio')
@
      \caption{Visualization of depth.ratio bias in relation of GC content (left), and resulting normalization effect (right).}
      \label{fig:GCfig}
\end{figure}

\section{Analyzing sequencing data with \texttt{sequenza}}

The R package \texttt{sequenza} offers an ensemble of functions and models that can be used to design customized workflows and analyses. Here we provide generic and most commonly used analysis steps. 

\begin{itemize}
\item Extract the relevant information from the raw \textit{seqz} file.
\item Fit the \textit{sequenza} model to infer cellularity and ploidy.
\item Apply the inferred parameters to estimate the copy number profile
\end{itemize}

\subsection{Extract the information from the \textit{seqz} file.}
The function \texttt{sequenza.extract} is designed to efficiently access the \textit{seqz} file and take care of normalization steps. The arguments enable customization of a set of actions listed below:
\begin{itemize}
\item binning depth ratio and B allele frequency in a desired window size (allowing a desired number of overlapping windows);
\item performing a fast, allele specific segmentation using the \texttt{copynumber} package\cite{BMCcopynumber};
\item filter mutations by frequency and noise.
\end{itemize}

<<label=sequenzaExtract, echo=TRUE, results=hide>>=
test <- sequenza.extract(data.file)
names(test)
@
After the raw data is processed, the size of the data is considerably reduced. Typically, the R object resulting from \texttt{sequenza.extract} can be stored as a file of a few megabytes, even for whole genome sequencing data. 

The result of this first step consists of a list of sub-lists. Each sub-list has information subdivided by chromosome. All lists share the same chromosome order.

\subsubsection{Plot chromosome view with mutations, BAF, depth ratio and segments}

Each chromosome can be visualized using the function \texttt{chromosome.view} as in Figure \ref{fig:3panelsPlot}. The same function can be used to visualize the data after the estimation of \textit{cellularity} and \textit{ploidy} parameters as in Figure \ref{fig:crhomViewWithCP}.

\begin{figure}[H]
      \centering
<<label=3panelsPlot,fig=TRUE,width=6,height=6>>=

chromosome.view(mut.tab = test$mutations[[1]], baf.windows = test$BAF[[1]], 
                ratio.windows = test$ratio[[1]], min.N.ratio = 1,
                segments = test$segments[[1]], main = test$chromosomes[1])
@ 
      \caption{Plots of mutant allele frequency (top), B allele frequency (middle) and depth ratio (bottom) vs. chromosome position.}
      \label{fig:3panelsPlot}
\end{figure}


\subsection{Inference of cellularity and ploidy}

After the raw data is processed, imported into R, and normalized, we can apply the parameter inference implemented in the package.
The function \texttt{sequenza.fit} performs the inference using the calculated B allele frequency and depth ratio of the obtained segments. The method can be explored in more detail by reading the manual pages for the function \texttt{baf.model.fit}.

<<label=sequenzaFit, echo=TRUE, eval=FALSE, results=hide>>=
CP.example <- sequenza.fit(test)
@

<<label=loadCP, echo=FALSE, results=hide>>=
data(CP.example)
@

The result is a list in a format \textit{list(ploidy, collularity, lpp)}, corresponding to the conventional format \textit{list(x, y, z)}, easily usable by standard graphical functions, such as \texttt{image}. However we provide functions to explore and better display the results, and to extract the point estimate and confidence intervals.

\subsection{Results of model fitting}

The last part of the workflow is to apply the estimated parameters. There is an all-in-one function that plots and saves the results, giving control on file names and output  directory:

<<label=sequenzaRes, echo=TRUE, eval=FALSE, results=hide>>=
sequenza.results(sequenza.extract = test, cp.table = CP.example,
                 sample.id = "Test", out.dir="TEST")
@

Although this standard way of presenting the result is appropriate for most situations, it is possible to create an alternative wrapper by using functions in the following sub-sections.

\subsubsection{Confidence intervals, confidence region and point estimate}

The object resulting from \texttt{sequenza.fit} has two vectors, \textit{ploidy} and \textit{cellularity}, indicating respectively the tested values of ploidy and cellularity, and a matrix, \textit{lpp}, with columns and rows corresponding to the previously indicated vectors, containing the estimated log posterior probability for the combinations of cellularity and ploidy.
Confidence intervals for these two parameters can be calculated using the function \texttt{get.ci}.
<<label=ConfIntCP, echo=TRUE>>=
cint <- get.ci(CP.example)
@

It is also possible to plot the posterior probability over the combinations of the two parameters, highlighting the point estimate and the confidence region.

\begin{figure}[H]
      \centering
<<label=CPplot,fig=TRUE,width=6,height=6>>=
cp.plot(CP.example)
cp.plot.contours(CP.example, add = TRUE, likThresh = c(0.95))
@ 
 \caption{Result from the inference over the defined range of cellularity and ploidy. Color intensity indicates the log posterior probability of corresponding cellularity/ploidy values.}
      \label{fig:cpPlot}
\end{figure}

By exploring the results for \textit{cellularity} and \textit{ploidy} separately, it is possible to draw the posterior probability distribution for each parameter. The information is returned by the \texttt{get.ci} function.
\begin{figure}[H]
      \centering
<<label=CPplotCI,fig=TRUE,width=6,height=6>>=
par(mfrow = c(2,2))
cp.plot(CP.example)
cp.plot.contours(CP.example, add = TRUE)
plot(cint$values.cellularity, ylab = "Cellularity",
     xlab = "posterior probability", type = "n")
select <- cint$confint.cellularity[1] <= cint$values.cellularity[,2] &
          cint$values.cellularity[,2] <= cint$confint.cellularity[2]
polygon(y = c(cint$confint.cellularity[1], cint$values.cellularity[select, 2], cint$confint.cellularity[2]), 
        x = c(0, cint$values.cellularity[select, 1], 0), col='red', border=NA)
lines(cint$values.cellularity)
abline(h = cint$max.cellularity, lty = 2, lwd = 0.5)  

plot(cint$values.ploidy, xlab = "Ploidy",
     ylab = "posterior probability", type = "n")
select <- cint$confint.ploidy[1] <= cint$values.ploidy[,1] &
          cint$values.ploidy[,1] <= cint$confint.ploidy[2]
polygon(x = c(cint$confint.ploidy[1], cint$values.ploidy[select, 1], cint$confint.ploidy[2]), 
        y = c(0, cint$values.ploidy[select, 2], 0), col='red', border=NA)
lines(cint$values.ploidy)
abline(v = cint$max.ploidy, lty = 2, lwd = 0.5)

@ 
      \caption{Plot of the log posterior probability with respective cellularity and ploidy probability distribution and confidence intervals.}
      \label{fig:cpPlotCI}
\end{figure}

\subsection{Call CNVs and mutations using the estimated parameters}

The point estimate value corresponds to the point of maximum posterior probability, detected after the confidence interval computation: 
<<label=seParam, eval=TRUE>>=
cellularity <- cint$max.cellularity
cellularity
ploidy <- cint$max.ploidy
ploidy
@
In addition we need to calculate the average normalized depth ratio, used to set a value for the baseline copy number. 
<<label=avgDepth, eval=TRUE>>=
avg.depth.ratio <- mean(test$gc$adj[, 2])
avg.depth.ratio
@

\subsubsection{Detect variant alleles (mutations)}

To detect variant alleles, we use a mutation frequency model that is implemented as the \texttt{mufreq.bayes} function:

<<label=mmufBayes, eval=TRUE>>=
mut.tab     <- na.exclude(do.call(rbind, test$mutations))
mut.alleles <- mufreq.bayes(mufreq = mut.tab$F,
                            depth.ratio = mut.tab$adjusted.ratio,
                            cellularity = cellularity, ploidy = ploidy,
                            avg.depth.ratio = avg.depth.ratio)

head(mut.alleles)
head(cbind(mut.tab[,c("chromosome","position","F","adjusted.ratio", "mutation")],
           mut.alleles))
@

The result consists of four values for every imputed mutation: \textit{CNn} is the provided copy number of the normal sample at the given position (default = 2); \textit{CNt} is the estimated copy number of the tumor at the given position; \textit{Mt} is the estimated numbers of alleles carrying the mutation; \textit{LPP} is the log posterior probability of the model fit.

\subsubsection{Detect copy number variations}

To detect copy number variations we use the model implemented in the function \texttt{baf.bayes}, with the estimated parameters of \textit{cellularity} and \textit{ploidy}:
<<label=bafBayes, eval=TRUE>>=
seg.tab     <- na.exclude(do.call(rbind, test$segments))
cn.alleles <- baf.bayes(Bf = seg.tab$Bf, depth.ratio = seg.tab$depth.ratio,
                        cellularity = cellularity, ploidy = ploidy,
                        avg.depth.ratio = avg.depth.ratio)
head(cn.alleles)
seg.tab <- cbind(seg.tab, cn.alleles)
head(seg.tab)
@
The result consists of four values for every imputed segment: \textit{CNt} is the estimated copy number of the tumor of the given segment; \textit{A} is the estimated number of A alleles; \textit{B} is the estimated number of B alleles; \textit{LPP} is the log posterior probability of the model fit.

\subsection{Visualize detected copy number changes and variant alleles}

To visualize the data after detection of CNV and variant alleles, one can use the \texttt{chromosome.view}. In order to draw the relative model points (and to evaluate how the estimated model fits the real data) additional information is needed beyond that in Figure \ref{fig:3panelsPlot}:
\begin{itemize}
\item Each segment must have the columns relative to the copy number variation calling. 
\item \textit{Cellularity} and \textit{ploidy} estimates.
\item Average normalized depth ratio.
\end{itemize}
\begin{figure}[H]
      \centering
<<label=chrViewWithCP,fig=TRUE,width=6,height=6>>=
chromosome.view(mut.tab = test$mutations[[3]], baf.windows = test$BAF[[3]], 
                ratio.windows = test$ratio[[3]],  min.N.ratio = 1,
                segments = seg.tab[seg.tab$chromosome == test$chromosomes[3],],
                main = test$chromosomes[3],
                cellularity = cellularity, ploidy = ploidy,
                avg.depth.ratio = avg.depth.ratio)
@ 
      \caption{Plots of mutant allele frequency (top), B allele frequency (middle) and depth ratio (bottom) for chromosome position. Horizontal dotted lines indicate expectation values for various copy number/allele states.}
      \label{fig:crhomViewWithCP}
\end{figure}

\subsubsection{Genome-wide view of the allele and copy number state}

\begin{figure}[H]
      \centering
<<label=genomeViewCNt, fig=TRUE, width=7, height=3>>=
genome.view(seg.cn = seg.tab, info.type = "CNt")
legend("bottomright", bty="n", c("Tumor copy number"),col = c("red"), 
       inset = c(0, -0.4), pch=15, xpd = TRUE)
@
      \caption{Genome-wide absolute copy number profile obtained from exome sequencing.}
      \label{fig:genomeViewCNt}
\end{figure}

\begin{figure}[H]
      \centering
<<label=genomeViewAB, fig=TRUE, width=7, height=3>>=
genome.view(seg.cn = seg.tab, info.type = "AB")
legend("bottomright", bty = "n", c("A-allele","B-allele"), col= c("red", "blue"), 
       inset = c(0, -0.45), pch = 15, xpd = TRUE)
@
      \caption{Genome-wide allele-specific copy number profile obtained from exome sequencing.}
      \label{fig:genomeViewAB}
\end{figure}


%\newpage
%%%% bibliography
\bibliographystyle{plain}
\bibliography{sequenza}

\end{document}

                