Metrics file not output for case

[seallard]

For some cases, the expected metrics file is not written.

Example of error log when running QC for a case. The files were written to Hasta a week earlier, so it seems reasonable to expect the metrics file to exist.

[09:11] [hiseq.clinical@hasta:/home/proj/production/microbial/results] [P_base]  $ cg workflow microsalt qc flexiblegannet
Called undefined __fields__ on HousekeeperAPI, please wrap
Called undefined __fields__ on HousekeeperAPI, please wrap
Called undefined __fields__ on HousekeeperAPI, please wrap
Called undefined __fields__ on HousekeeperAPI, please wrap
Called undefined __fields__ on HousekeeperAPI, please wrap
Called undefined __fields__ on HousekeeperAPI, please wrap
Called undefined __fields__ on HousekeeperAPI, please wrap
Performing QC on case flexiblegannet
Traceback (most recent call last):
  File "/home/proj/production/bin/miniconda3/envs/P_cg/bin/cg", line 8, in <module>
    sys.exit(base())
             ^^^^^^
  File "/home/proj/production/bin/miniconda3/envs/P_cg/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/proj/production/bin/miniconda3/envs/P_cg/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/proj/production/bin/miniconda3/envs/P_cg/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/proj/production/bin/miniconda3/envs/P_cg/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/proj/production/bin/miniconda3/envs/P_cg/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/proj/production/bin/miniconda3/envs/P_cg/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/proj/production/bin/miniconda3/envs/P_cg/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/proj/production/bin/miniconda3/envs/P_cg/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/proj/production/bin/miniconda3/envs/P_cg/lib/python3.11/site-packages/cg/cli/workflow/microsalt/base.py", line 224, in qc_microsalt
    analysis_api.quality_checker.quality_control(metrics_file_path)
  File "/home/proj/production/bin/miniconda3/envs/P_cg/lib/python3.11/site-packages/cg/meta/workflow/microsalt/quality_controller/quality_controller.py", line 46, in quality_control
    quality_metrics: QualityMetrics = MetricsParser.parse(case_metrics_file_path)
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/proj/production/bin/miniconda3/envs/P_cg/lib/python3.11/site-packages/cg/meta/workflow/microsalt/metrics_parser/metrics_parser.py", line 10, in parse
    data = read_json(file_path)
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/proj/production/bin/miniconda3/envs/P_cg/lib/python3.11/site-packages/cg/io/json.py", line 8, in read_json
    with open(file_path, "r") as file:
         ^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/home/proj/production/microbial/results/ACC14988_2024.7.8_4.15.51/ACC14988.json'

[seallard]

Frequency: this seems to occur frequently, at least weekly according to production.
Production can usually work around it by manually inspecting the individual QC files in the case. However, for this specific instance - those files were not generated either and the workaround was to run

>  conda activate P_microSALT
>  microSALT utils finish /home/proj/production/microbial/queries/flexiblegannet.json --input /home/proj/production/microbial/results/ACC14988_2024.7.8_4.15.51/
>  cg workflow mircosalt qc flexiblegannet

Additional observations

The case was still set to running in status db, despite all slurm jobs having been completed for it.

[seallard]

I could not identify a cause and sys devs are not responsible for maintaining pipelines, so I'll leaves this.
A workaround exists.