Check consistency of settings at different steps of the pipeline

[aleberti]

To keep track of what was discussed in #148

There were some that involved pathological cases with people playing with tel_id numbering, but this is not related to ctapipe0.19 compatibility so we can look into this later on with a new PR. Concerning this test that you implemented, I only saw the one checking consistency between different subarray information if multiple files are read at once, while we were discussing that the telescope numbering should be set at the first analysis step and then it should not be changed by new settings in input card, or there should be a check that the settings that are used in the input card do not clash with what is read from the subarray, but again, I think we can do it properly in another PR. Let's merge this one :-).

[aleberti]

@jsitarek I see here different possibilities:

• in the first steps of the pipeline (magic_calib_to_dl1 and lst1_magic_mc_dl0_to_dl1) we write the subarray configuration, then at later steps we read back the subarray information and the IDs from the config file to check that they match. In general this may not 100% fail proof because for example, in MC with 4LSTs, the subarray telescope descriptions do not have specific names like "LST-1", "LST-2" and so on, but just a generic "LST". So in this case I can just check that in the subarray there are the IDs specified in the config files and that the telescope type is correct
• the other thing that can be done, is to write out the dictionary with the telescope IDs from the config file to the HDF5 files created in magic_calib_to_dl1 and lst1_magic_mc_dl0_to_dl1. Then this would be checked in the next steps. So this would ensure that the IDs are not changed in the config file in following steps of the processing

[jsitarek]

I like the second solution. The only problem is that once we do such a change it will not be backward compatible, and the old files will not be readable with the new code. But since we still did not produce any major production with ctapipe0.19-compatible branch I think it is fine.

[Elisa-Visentin]

We can keep backward compatibility if we read the dictionary with the IDs only if it has been written to the file (e.g., an if clause on the MCP version or a try-except)

[Elisa-Visentin]

Any news about that? What about using the MCP version and, if version>= xxx check as in option 2, else check as in 1? This should not cause any backward incompatibility (we add some info in the h5 since v xxx, but we retrieve it only if v xxx, so it should never fail, hopefully)

[jsitarek]

Hi @Elisa-Visentin
I think we do not store the MCP version with which the file was processed, so it would have to go through try/except clause

[Elisa-Visentin]

@jsitarek: unfortunately yes (we store it only in dl3), but at least it could be checked in case it is stored, independently of the version. For the files in which no Tel. configuration is stored, no checks can be performed and the user must take care of not changing the tel_ids during the analysis