BBMerge Guide
Written by Brian Bushnell
Last updated December 11, 2015

BBMerge is designed to merge two overlapping paired reads into a single read.  For example, a 2x150bp read pair with an insert size of 270bp would result in a single 270bp read.  This is useful in amplicon studies, as clustering and consensus are far easier with single reads than paired reads, and also in assembly, where longer reads allow the use of longer kmers (for kmer-based assemblers) or fewer comparisons (for overlap-based assemblers).  And in either case, the quality of the overlapping bases is improved.  BBMerge is also capable of error-correcting the overlapping portion of reads without merging them, as well as merging nonoverlapping reads, if enough coverage is available.  BBMerge is the fastest and by far the most accurate overlap-based read merger currently in existence.

BBMerge's parameters are described in its shellscript (bbmerge.sh).  This file provides usage examples of various common tasks.


*Notes*


Memory:

BBMerge has 2 shellscripts, bbmerge.sh and bbmerge-auto.sh.  They are equivalent except for memory usage; so if you override memory with the -Xmx flag, they are equivalent.  bbmerge.sh is designed for overlap-based merging only, and uses a fixed 1GB of RAM (though it can function with much less than that).  bbmerge-auto.sh is designed for merging overlapping and non-overlapping reads, and attempts to grab all available physical memory, which is needed for storing kmers.


Output streams:

BBMerge supports "out" (aka "outm" or "outmerged") and "outu" ("outunmerged").  Reads that are merged, or mergeable, go to out, and the rest go to outu.  There is a "join" flag (default true) that controls whether mergeable reads get merged.  If it is set to false, mergeable reads will be written interleaved to out.  All output streams are optional.


Threads and speed:

BBMerge is multithreaded and scales linearly with the number of processor cores, so it's best to let it automatically use all of them.  You can restrict the number of worker threads with the "t" flag if you are working on a shared node.  To achieve the maximal speed on a system with many (20+) cores, BBMerge should be fed two files (using in1 and in2) rather than a single interleaved file.


JNI acceleration:

BBMerge has an optional C component (written by Jonathan Rood) which will accelerate merging by approximately 20%.  This can be activated with the "jni" flag, but it must first be compiled.  For details on compiling it, see /bbmap/jni/README.txt


Strictness:

BBMerge has a lot of settings controlling merging stringency, such as maxratio, ratiomargin, entropy, efilter, etc.  Advanced users should feel free to tune these as needed.  But it's a lot simpler to use the predefined strictness levels which adjust the specific settings according to the results of extensive benchmarking.  To use a predefined strictness level, simply add a flag like "loose" (you don't need to add one for "default").  The predefined strictness levels, from strictest to loosest:
xstrict, ustrict, vstrict, strict, default, loose, vloose, uloose, xloose
Stricter settings have lower merge rates and fewer false positives; looser settings have higher merge rates and more false positives.  Loose settings are generally not necessary except with low-quality data (which often happens in low-diversity amplicon sequencing using long reads).  A false-positive means a read pair that merged with the wrong overlap length - these can cause problems in assembly or clustering (leading to spurious clusters).  However, at any level of strictness, BBMerge has by far the lowest false-positive rate of any read merger, in my testing.


When not to use:

If you run BBMerge, and under, say, 15% of the reads merge, even at very loose stringency, it's probably a waste of time to merge - you'll just make the workflow more complicated, and possibly get a lot of false-positives.  Also, don't try to merge single-ended libraries or long-mate-pair libraries that are not in an "innie" orientation.


*Usage Examples*

Basic merging:
bbmerge.sh in=reads.fq out=merged.fq outu=unmerged.fq ihist=ihist.txt

This will merge the reads by overlap.  If no best overlap is found, the pair will go to outu; otherwise, the reads will be merged and sent to out.  After finishing, an insert size histogram will be written to ihist.txt.


Overlap-based error-correction:
bbmerge.sh in=reads.fq out=corrected.fq ecco mix

This will correct reads that overlap, rather than merging them.  Where the two reads agree, the quality score will be increased; where they disagree, the score will be reduced, and the base call will be changed to the base with the higher quality.  If the bases differ and the scores are equal, the base will be replaced with N.


Merging of nonoverlapping reads:
bbmerge-auto.sh in=reads.fq out=merged.fq outu=unmerged.fq ihist=ihist.txt ecct extend2=20 iterations=5

This will attempt to merge each pair by overlap.  If unsuccessful, both reads will be error-corrected using Tadpole, and then merging will be tried again.  If still unsuccessful, both reads will be extended by 20bp, then merging will be attempted again.  This will repeat up to 5 times, or until neither of the reads can be extended any more due to a branch or dead-end in the kmer graph.  If the reads are not merged, all of the changes are undone and the original pair will be sent to outu.  "extend2=20 iterations=5" will extend each read by up to 100bp, which increases the maximum insert sizes that can be merged by 200bp.  So, for example, a 2x150bp library can normally only merge inserts up to around 290bp; this would extend that capability to 490bp, and the middle would be filled in with assembled bases.


Discovering adapter sequences:
bbmerge.sh in=reads.fq outa=adapters.fa

This will report the consensus adapter sequences of pairs with insert size shorter than read length.
