Compress .allValidPairs and .txt files #220
Description
Description of feature
Hello,
given the output size of .allValidPairs and .txt files, and their high compression ratio, I suggest adding a step in the pipeline to compress them.
I specifically refer to the files in the following folders:
- results/hicpro/valid_pairs
- results/contact_maps/txt
Here is an example of the storage saving I got by compressing the files of a run:
| Path | Original Size (du-shc) | Compressed size (gzip -9 ) |
|---|---|---|
| results/hicpro/valid_pairs/*allValidPairs | 28G | 6.2G (-78%) |
| results/contact_maps/txt/*txt | 2.8G | 819M (-71%) |
Thank you!