Fixup some execution issues

Author: jeromekelleher

docs/Makefile

@@ -3,6 +3,9 @@ PYPATH=$(shell pwd)/../
 B2Z_VERSION:=$(shell PYTHONPATH=${PYPATH} \
    python3 -c 'import bio2zarr; print(bio2zarr.__version__.split("+")[0])')
 
+
+# FIXME this is all very fragile and needs to be rewritten. 
+# https://github.com/sgkit-dev/bio2zarr/issues/238
 CASTS=_static/vcf2zarr_convert.cast\
 	_static/vcf2zarr_explode.cast
 

docs/vcf2zarr/tutorial.md

@@ -121,7 +121,7 @@ head -n 20 sample.schema.json
 ```
 
 We've displayed the first 20 lines here so you can get a feel for the JSON format.
-The [jq](https://jqlang.github.io/jq/) provides a useful way of manipulating
+The [jq](https://jqlang.github.io/jq/) tool provides a useful way of manipulating
 these schemas. Let's look at the schema for just the ``call_genotype``
 field, for example:
 
@@ -158,6 +158,15 @@ vcf2zarr mkschema sample.icf \
 ```
 Then we can use the updated schema as input to ``encode``:
 
+
+<!-- FIXME shouldn't need to do this, but currently the execution model is very --> 
+<!-- fragile. -->
+<!-- https://github.com/sgkit-dev/bio2zarr/issues/238 -->
+```{code-cell}
+:tags: [remove-cell]
+rm -fR sample_noHQ.vcz
+```
+
 ```{code-cell}
 vcf2zarr encode sample.icf -s sample_noHQ.schema.json sample_noHQ.vcz
 ```
@@ -167,95 +176,9 @@ We can then ``inspect`` to see that there is no ``call_HQ`` array in the output:
 vcf2zarr inspect sample_noHQ.vcz
 ```
 
-
 ## Large
 
 
 
-## Parallel encode/explode
-
-
-## Common options
-
-```
-$ vcf2zarr convert <VCF1> <VCF2> <zarr>
-```
-
-Converts the VCF to zarr format.
-
-**Do not use this for anything but the smallest files**
-
-The recommended approach is to use a multi-stage conversion
-
-First, convert the VCF into the intermediate format:
-
-```
-vcf2zarr explode tests/data/vcf/sample.vcf.gz tmp/sample.exploded
-```
-
-Then, (optionally) inspect this representation to get a feel for your dataset
-```
-vcf2zarr inspect tmp/sample.exploded
-```
-
-Then, (optionally) generate a conversion schema to describe the corresponding
-Zarr arrays:
-
-```
-vcf2zarr mkschema tmp/sample.exploded > sample.schema.json
-```
-
-View and edit the schema, deleting any columns you don't want, or tweaking
-dtypes and compression settings to your taste.
-
-Finally, encode to Zarr:
-```
-vcf2zarr encode tmp/sample.exploded tmp/sample.zarr -s sample.schema.json
-```
-
-Use the ``-p, --worker-processes`` argument to control the number of workers used
-in the ``explode`` and ``encode`` phases.
-
-## To be merged with above
-
-The simplest usage is:
-
-```
-$ vcf2zarr convert [VCF_FILE] [ZARR_PATH]
-```
-
-
-This will convert the indexed VCF (or BCF) into the vcfzarr format in a single
-step. As this writes the intermediate columnar format to a temporary directory,
-we only recommend this approach for small files (< 1GB, say).
-
-The recommended approach is to run the conversion in two passes, and
-to keep the intermediate columnar format ("exploded") around to facilitate
-experimentation with chunk sizes and compression settings:
-
-```
-$ vcf2zarr explode [VCF_FILE_1] ... [VCF_FILE_N] [ICF_PATH]
-$ vcf2zarr encode [ICF_PATH] [ZARR_PATH]
-```
-
-The inspect command provides a way to view contents of an exploded ICF
-or Zarr:
-
-```
-$ vcf2zarr inspect [PATH]
-```
-
-This is useful when tweaking chunk sizes and compression settings to suit
-your dataset, using the mkschema command and --schema option to encode:
-
-```
-$ vcf2zarr mkschema [ICF_PATH] > schema.json
-$ vcf2zarr encode [ICF_PATH] [ZARR_PATH] --schema schema.json
-```
 
-By editing the schema.json file you can drop columns that are not of interest
-and edit column specific compression settings. The --max-variant-chunks option
-to encode allows you to try out these options on small subsets, hopefully
-arriving at settings with the desired balance of compression and query
-performance.