QA only accepts local input files. Should QA allow remote input files?

[nemo794]

Currently, QA is only designed for input files that are available on local filesystem.

Question for discussion: Should QA be updated to allow remote input files?

The possible input files are:

• L1/L2 science HDF5 granule
• (RSLC and GSLC) Corner reflector CSV, for AbsCal and PTA tools
• (GSLC) DEM file, for PTA tool

Note: While a one-size-fits-all decision would be nice, each file could be considered independently. E.G. a remote DEM is ok because that it passed directly into ISCE3, but maybe the corner reflector file must be local.

Action Items:
1) In this Issue, let's discuss and determine which (if any) input files should be allowed to be remote.
2) Determine plan to update QA code
2) Update QA code

Background:
For mission operations, GSLC SAS and GSLC QA SAS must be able to share the same runconfig, and the same dem_file parameter.

A user attempted to run GSLC QA with the DEM runconfig parameter set to:
/vsis3/<s3 bucket>/inputs/rslc_antimeridian_fiji/dem_Antimeridian_Fiji.vrt.
In general, files at /vsi... are remote rasters as explained here by GDAL.)
When this remote DEM file was used by the ISCE3 GSLC SAS, this did not raise any issues because ISCE3 passed the path directly to GDAL, and GDAL could work with that remote file.

However, QA is currently coded to expect the DEM file to be local. QA checks that the DEM file is valid via if not os.path.isfile(filepath). When the /vsi... file is provided, QA raises the error: "dem_file is not a valid file: /vsis3//inputs/rslc_antimeridian_fiji/dem_Antimeridian_Fiji.vrt", because QA could not find the file on the local filesystem.

Since the DEM is only (currently) used by ISCE3 / GDAL, then probably QA should not aggressively enforce unnecessary constraints on the input.

As a quick fix for the DEM file, we could simply replace these lines with a check that the parameter provided is a string or None, and then let GDAL raise the error from within ISCE3 if it cannot read the file. But, based on the outcome of this discussion, there could be a more robust resolution.

Why has this issue only just appeared? Likely because developers and SDS always stage a local DEM file, which QA could find. Which means this bug will likely not matter during operations, but does impact iScale and other users.

[nemo794]

Note: This Discussion replaces Issue #26 . Once a decision is reached here, we can re-open that Issue.