Harmony 1980 - Support for external validation services

[indiejames]

Jira Issue ID

HARMONY-1980

Description

Adds support for external validation for services.

NOTE: When we add an actual external validation for a service we will need to add an environment variable for it and refer to this in services.yml. Since we don't have any yet I have not done this.

Local Test Steps

1. add the following to the uat harmony-service-example config in services.yml just below the umm_s entry:

    external_validation_url: 'http://localhost:3333'

2. Run the following bash code to start up a simple server that will simulate an 'OK' validation:

while true; do { echo -ne "HTTP/1.1 200 OK\r\nContent-Length: 2\r\nContent-Type: text/plain\r\n\r\nOK"; } | nc -l 127.0.0.1 3333; done

3. Run the following query:

http://localhost:3000/C1233800302-EEDTEST/ogc-api-coverages/1.0.0/collections/red_var,blue_var,green_var/coverage/rangeset/?subset=lat(20%3A60)&subset=lon(-140%3A-50)&outputCrs=EPSG%3A31975&format=image%2Fpng&maxResults=1

This should succeed.

4. Kill the validation server started on line 2 and run the following code to simulate a failure:

while true; do { echo -ne "HTTP/1.1 403 Forbidden\r\nContent-Length: 9\r\nContent-Type: text/plain\r\n\r\nForbidden"; } | nc -l 127.0.0.1 3333; done

5. Run the query from 3 again. You should see the following error:

6. Kill the validation server and run the query from 3 again (simulating the case where harmony can't reach the validation endpoint). You should see the following error:

{
	"code": "harmony.ServerError",
	"description": "Error: Internal server error"
}

To test in hiab change the line from step 1 to

external_validation_url: 'http://host.docker.internal:3333'

Then rebuild the harmony image to pick up latest code changes. Then after running hiab repeat steps 2-6.

PR Acceptance Checklist

• Acceptance criteria met
• Tests added/updated (if needed) and passing
• Documentation updated (if needed)
• Harmony in a Box tested? (if changes made to microservices or new dependencies added)

[chris-durbin]

Looks great - tested successfully and really liked the bash server that printed out the messages received. My only comments are logging related.

[chris-durbin]

I'm wondering if we should try to future proof it to allow for multiple validations and different types of validations. e.g.

validations:
  - url: !Env ${FOO_ENDPOINT}
  - docker: !Env ${FOO_IMAGE}
  - code: /app/validations/foo.ts

It's probably fine to keep it just a single URL for now and change it if needed in the future.

[indiejames]

Yeah, I think we will keep it simple for now and support those use cases as needed.

[chris-durbin]

Change this to log something like Error calling validation endpoint: ${url}.

[indiejames]

Done.

[chris-durbin]

Remove this log line (it's displaying things we do not want in logs).

[indiejames]

Done.

[chris-durbin]

We should log the response body and status code the validation URL returned - I didn't see either in the logs.

[indiejames]

Done.

[chris-durbin]

I'd like to add an info level timing message for calling the validation URL (see the other places we log durationMs in the code).

[indiejames]

Done.

[ygliuvt]

nit: log the response status here as well.

[indiejames]

Done.

[ygliuvt]

Tested successfully locally.