parsing and resolving resource parameters in ocrd_network

[bertsky]

Another problem is with how parameter resources get resolved now. In the CLI setting, we allowed an ambiguity between JSON literals and file names, so the latter would have to be resolved to absolute paths (for the various resource locations), which in turn obviously dependend on the processor class (because of the custom module location):

core/src/ocrd/decorators/__init__.py

Lines 95 to 103 in 4d33491

	if 'parameter' in kwargs:
	# Disambiguate parameter file/literal, and resolve file
	def resolve(name):
	try:
	return processor.resolve_resource(name)
	except ResourceNotFoundError:
	return None
	kwargs['parameter'] = parse_json_string_or_file(*kwargs['parameter'],
	resolve_preset_file=resolve)

But in the network setting, where parameters can be added to the processing or workflow request, no such resolution (from resource name to resource path) takes place –

• On the server side, there is only a parameter validation step
  

  core/src/ocrd_network/server_utils.py

  Line 220 in 4d33491

  report = ParameterValidator(ocrd_tool).validate(dict(job_input.parameters))

• On the client side, there is only the JSON string vs file disambiguation, but no prior resolution.
  

  core/src/ocrd_network/cli/client.py

  Line 142 in 4d33491

  req_params["parameters"] = set_json_key_value_overrides(parse_json_string_or_file(*parameter), *parameter_override)

Did we abandon the entire mechanism, or am I missing something?

Originally posted by @bertsky in #1303 (comment)

[bertsky]

So what does processor.resolve_resource(name) actually do?

Well, …

core/src/ocrd/processor/base.py

Lines 937 to 938 in 47b77aa

	ret = list(filter(exists, list_resource_candidates(executable, val,
	cwd=cwd, moduled=self.moduledir)))

… thus it uses ocrd_utils.list_resource_candidates() with the processor's self.moduledir. The latter (being installation-dependent) is only known to the processor's codebase, so it requires processor instantiation. That's why outside the Processor class, we have ocrd_utils.get_moduledir() for this, which tries to read the install-time ocrd-all-module-dir.json file (which we deploy in both in ocrd/all fat images and in all continuosly deployed thin/per-module Docker images), and only then falls back on a CLI instantiation (--dump-module-dir).

So what does that mean for ocrd_network?

Well, since even ocrd-all-module-dir.json is installation-dependent (and thus also processor-dependent), we need some runtime component (installed) on the processor side which can answer our list_resource_candidates() in a similar way. And that would have to be the new Resource Manager Server!

Which already uses the local ResourceManager's new build_resource_dest_dir() for the download case, so it could easily be extended to use the same for the resolve case.

Which BTW is not the same as the list_installed() case, unless we make the latter return only relative paths in the network setting. Currently, it is wrongly (!) implemented via list_available():

core/src/ocrd_network/resource_manager_server.py

Lines 102 to 107 in 47b77aa

	async def list_installed_resources(self, executable: Any = None):
	if executable == '*':
	message = f"'*' is not an acceptable executable name! Try with a specific executable."
	self.log.error(message)
	raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=message)
	result = self.resmgr_instance.list_available(executable)

Regardless how we do that concretely, once we resolve parameter values as potential relative path names via RMS requests, we can then (and only then) correctly validate the parameters for the processing or workflow request. IMO this does not need to be on the client side though. And it does not at all make sense to translate the JSON literal vs. path name ambiguity to the network clients as well: this was only a convenience feature for the CLI user. In contrast, in ocrd_network, we should be clear as to what is a JSON literal, what is a parameter file to parse on the client side, and what is a path name to resolve on the worker side (but also pre-resolve on the server side for early validation).