Updating the docs for counts and for format ahead of v1.2.5

Author: andrewdavidsmith

docs/content/counts.md

@@ -2,7 +2,7 @@
 
 ## Synopsis
 ```console
-$ dnmtools counts [OPTIONS] -c <chroms> <input.sam>
+$ dnmtools counts [OPTIONS] -c <chroms> <input.bam>
 ```
 
 ## Description
@@ -18,7 +18,7 @@ pluripotent mammalian cells such as embryonic stem cells. And possibly
 whatever cells you are studying. The output of `counts` serves as the
 input for many downstream analyses.
 
-The input mapped reads file (`input.sam`) is in SAM/BAM format. The
+The input mapped reads file (`input.bam`) is in SAM/BAM format. The
 reads should be sorted so those mapping to the same chromosome are
 consecutive in the file. Duplicate reads should be probably be
 [removed](../uniq) first, but that depends on your data.
@@ -40,12 +40,12 @@ $ dnmtools counts -c /path/to/genome.fa -o output.meth input.sam
 ```
 
 The argument `-c` gives the name of a FASTA file containing all
-chromosome sequences or a directory that contains one FASTA format
-file for each chromosome. By default `counts` identifies these
-chromosome files by the extension `.fa`. Importantly, the "name" line
-in each chromosome FASTA file must begin with the character `>`
-followed immediately by the same name that identifies that chromosome
-in the SAM output (the `.sam` files). An example of the output and
+chromosome sequences (as of v1.2.5, a directory of separate files is
+no longer supported). Importantly, the "name" line in each chromosome
+FASTA file must begin with the character `>` followed immediately by
+the same name that identifies that chromosome in the SAM output (the
+`.bam` files). If you use the same FASTA format file you used to map
+the reads, everything should be fine. An example of the output and
 explanation of each column follows:
 ```txt
 chr1  1869  +  CCG  0         1
@@ -142,6 +142,24 @@ which is not useful unless commands are piped.
 Reference genome file, which must be in FASTA format. This is
 required.
 
+```txt
+-t, -threads
+```
+The number of threads to use. This is only really helpful if the input
+is BAM (not very helpful for SAM), and the output is to be zipped (see
+`-z` below). These threads will help decompress the BAM input and will
+help compress the gzip format output. If only one of these conditions
+holds, using more threads can still help. Because `counts` spends most
+of its computing time processing reads sequentially, there are
+diminishing returns for specifying too many threads.
+
+```txt
+-z, -zip
+```
+The output should be zipped (in gzip format). This is not deduced by
+the filename, but specifying this argument should be accompanied by
+using a `.gz` filename suffix for the output.
+
 ```txt
 -n, -cpg-only
 ```

docs/content/format.md

@@ -3,7 +3,7 @@
 ## Synopsis
 
 ```shell
-$ dnmtools format [OPTIONS] -f <mapper>  <input.sam>
+$ dnmtools format [OPTIONS] -f <mapper>  <input.bam> [output.bam]
 ```
 
 ## Description
@@ -16,12 +16,29 @@ important to quantify methylation, as fragments that overlap must
 count the overlapping bases only once and must be treated as
 originating from the same allele. These can be ensured by merging them
 into a single entry.  SAM/BAM files generated by abismal, Bismark and
-BSMAP can be formatted using the `format` command. An example use of
-this command to format a mapped reads file is:
+BSMAP can be formatted using the `format` command.
+
+An example use of this command to format a mapped reads file is:
 ```shell
-$ dnmtools format -f abismal -o input-formatted.sam input.sam
+$ dnmtools format -f abismal input.bam output.sam
 ```
 Above, the file `input.sam` would have been generated by `abismal`.
+The file `output.bam` is the output, and an output file is required
+here unless the `-stdout` argument is specified (see below). Another
+example:
+```shell
+$ dnmtools format -f abismal -t 8 -B input.bam output.bam
+```
+This will use 8 threads because of the `-t 8` and will produce output
+in BAM format because of the `-B` flag (not the filename of the
+output).
+
+*Note* As of dnmtools v1.2.5, there is no longer a "buffer size"
+argument. This introduced arbitrary behavior. Now `format` assumes
+reads are sorted by read name, which should ensure mates in paired-end
+sequencing are consecutive in the file. No "buffer" is needed, and
+data that does not conform is more easily detected, making this tool
+more easily detect improperly formatted input.
 
 ## Options
 
@@ -32,33 +49,63 @@ This option indicates the format of the input SAM file, corresponding
 to the mapper that generated it (options: abismal, bsmap, bismark).
 
 ```txt
--o, -output
+ -t, -threads
+```
+The number of threads to use. These threads are used for I/O, and are
+most helpful when the input and output are both BAM, where the threads
+can really speed things up.
+
+```txt
+ -B, -bam
 ```
-The name of the output file. The output will be in SAM format. By
-default this is standard output.
+The output is in BAM format. This is an option to help prevent
+accidentally writing BAM format to the terminal or through a pipe that
+expects plain text, e.g., SAM.
 
 ```txt
--s, -suffix
+ -stdout
+```
+Write the output to standard out. This is not done by default even
+without an output file given, because of the danger of writing BAM to
+the terminal or through a pipe unexpectedly. It is possible to write
+BAM redirected or through a pipe, but the `-stdout` argument is
+required.
+
+```txt
+-s, -suff
 ```
 The length of the suffix for read names, which indicates whether the
-read is from end 1 or end 2 (default: 1).
+read is from end 1 or end 2 for paired-end reads. If this is not
+specified, but the data is paired end (i.e., the flag `-single-end` is
+not used; see below), then the length of this suffix is inferred.
+
+```txt
+-single-end
+```
+Using this argument tells `format` not to look for mates to merge as a
+single fragment. The default assumption is that data is paired-ended
+and that mates are consecutive in the input.
 
 ```txt
 -L, -max-frag
 ```
 The maximum allowed insert size in base-pairs (default:
-10000). Normally this parameter is set at the mapping step, but
-`format` can also reject reads that are in opposing strands in the
-same chromosome but map more than "max-frag" bases apart.
+unlimited). Normally this parameter is determined during read mapping,
+but `format` can also reject reads that are in opposing strands in the
+same chromosome but map more than this many bases apart.
 
 ```txt
--B, -buf-size
+-F, -force
 ```
-Maximum buffer size (default: 10000). This is the maximum
-number of reads retained before mates are no longer seeked for
-reads. If more than "buf-size" reads are not a proper mate of a given
-read, the read is printed as-is and reported as single-end. This value
-has no effect if the input is single-end.
+This option "forces" the `format` command to process paired-end reads
+even if it is unable to detect mates. Without this argument, failure
+to detect mates will cause `format` to terminate. This option is
+useful, for example, if the reads were paired-ended, but the second
+end is of such low quality that only reads from the first end were
+mapped. In a data analysis pipeline, it might not be apparent that one
+of two ends failed entirely, so providing this option can help. If you
+are only analyzing a small number of data sets, you probably want to
+be made aware of this problem rather than force it to be ignored.
 
 ```txt
 -v, -verbose