Interoperability with other BCF implementations

[jmarshall]

We need to ensure that BCF files produced by Picard and other tools can be read by htslib/bcftools/etc, and also vice versa.

Writing BCF for consumption by Picard-based tools

First generate some test files based on bcftools/test/ex2.vcf:

bcftools view -ob -n ex2-bcftools.bcf test/ex2.vcf
bcftools view -ou -n ex2-bcftools.u.bcf test/ex2.vcf

picard VcfFormatConverter REQUIRE_INDEX=false I=ex2-bcftools.u.bcf O=fred.vcf

This produces fred.vcf with equivalent contents to the original test/ex2.vcf (provided the input file has no instances of BCFv2.2-specific sentinels), as expected.

picard VcfFormatConverter REQUIRE_INDEX=false I=ex2-bcftools.bcf O=fred.vcf

This currently produces

Exception in thread "main" org.broad.tribble.TribbleException$MalformedFeatureFile: Unable to parse header with error: Input stream does not contain a BCF encoded file; BCF magic header info not found, at record 0 with position 0:, for input source: /Users/jm18/bcftools/ex2-bcftools.bcf

as it seems that Picard currently only accepts what we call uncompressed-BCF.

Reading BCF files produced by Picard-based tools

This produces what we call an uncompressed BCF file, which should be autodetected by hts_open() and bcftools:

picard VcfFormatConverter REQUIRE_INDEX=false INPUT=test/ex2.vcf OUTPUT=ex2-picard.bcf

However bcftools refuses to read it:

bcftools view ex2-picard.bcf

[vcf.c:529 bcf_hdr_read] invalid BCF2 magic string: only BCFv2.2 is supported.
Fail to read VCF/BCF header: ex2-picard.bcf

The expected result is that we read ex2-picard.bcf successfully.