Adds support for translating one URI into another (#682)

* Adds support for translating one URI into another

* Adds documentation about URITranslator.

---------

Co-authored-by: Faron Dutton <[email protected]>
Co-authored-by: fdutton <fdutton@noreply>

Author: 3 people

doc/schema-map.md

@@ -1,24 +1,141 @@
 While working with JSON schema validation, we have to use external references sometimes. However, there are two issues to have references to schemas on the Internet. 
 
 * Some applications are running inside a corporate network without Internet access. 
-* Some of the Internet resources are not reliable
+* Some of the Internet resources are not reliable.
+* A test environment may serve unpublished schemas, which are not yet available at the location identified by the payload's `$schema` property.
 
 One solution is to change all the external reference to internal in JSON schemas, but this is error-prone and hard to maintain in a long run. 
 
 A smart solution is to map the external references to internal ones in a configuration file. This allows us to use the resources as they are without any modification. In the JSON schema specification, it is not allowed to use local filesystem resource directly. With the mapping, we can use the local resources without worrying about breaking the specification as the references are still in URL format in schemas. In addition, the mapped URL can be a different external URL, or embbeded within a JAR file with a lot more flexibility. 
 
 Note that when using a mapping, the local copy is always used, and the external reference is not queried.
 
-### Usage
+### URI Translation
 
-Basically, you can specify a mapping in the builder. For more details, please take a look at the test cases and the [PR](https://github.com/networknt/json-schema-validator/pull/125). 
+Both `SchemaValidatorsConfig` and `JsonSchemaFactory` accept one or more `URITranslator` instances. A `URITranslator` is responsible for providing a new URI when the given URI matches certain criteria.
 
+#### Examples
 
-### Real Example
+Automatically map HTTP to HTTPS
 
-https://github.com/JMRI/JMRI/blob/master/java/src/jmri/server/json/schema-map.json
+```java
+SchemaValidatorsConfig config = new SchemaValidatorsConfig();
+config.addUriTranlator(uri -> {
+    if ("http".equalsIgnoreCase(uri.getScheme()) {
+        try {
+            return new URI(
+                "https",
+                uri.getUserInfo(),
+                uri.getHost(),
+                uri.getPort(),
+                uri.getPath(),
+                uri.getQuery(),
+                uri.getFragment()
+            );
+        } catch (URISyntaxException x) {
+            throw new IllegalArgumentException(x.getMessage(), x);
+        }
+    }
+    return uri;
+});
+```
 
-In case you provide the schema through an `InputStream` or a `String` to resolve `$ref` with URN (relative path), you need to provide the `URNFactory` to the `JsonSchemaFactory.Builder.
-URNFactory` interface will allow you to resolve URN to URI.
+Map a public schema to a test environment
+
+```java
+SchemaValidatorsConfig config = new SchemaValidatorsConfig();
+config.addUriTranlator(uri -> {
+    if (true
+        && "https".equalsIgnoreCase(uri.getScheme()
+        && "schemas.acme.org".equalsIgnoreCase(uri.getHost())
+        && (-1 == uri.getPort() || 443 == uri.getPort())
+    ) {
+        try {
+            return new URI(
+                "http",
+                uri.getUserInfo(),
+                "test-schemas.acme.org",
+                8080,
+                uri.getPath(),
+                uri.getQuery(),
+                uri.getFragment()
+            );
+        } catch (URISyntaxException x) {
+            throw new IllegalArgumentException(x.getMessage(), x);
+        }
+    }
+    return uri;
+});
+```
+
+Replace a URI with another
+
+**Note:**
+This also works for mapping URNs to resources.
+
+```java
+SchemaValidatorsConfig config = new SchemaValidatorsConfig();
+config.addUriTranlator(URITranslator.map("https://schemas.acme.org/Foo", "classpath://Foo");
+```
+
+### Precedence
+
+Both `SchemaValidatorsConfig` and `JsonSchemaFactory` accept multiple `URITranslator`s and in general, they are evaluated in the order of addition. This means that each `URITranslator` receives the output of the previous translator. For example, assuming the following configuration:
+
+```
+SchemaValidatorsConfig config = new SchemaValidatorsConfig();
+config.addUriTranlator(uri -> {
+    if ("http".equalsIgnoreCase(uri.getScheme()) {
+        try {
+            return new URI(
+                "https",
+                uri.getUserInfo(),
+                uri.getHost(),
+                uri.getPort(),
+                uri.getPath(),
+                uri.getQuery(),
+                uri.getFragment()
+            );
+        } catch (URISyntaxException x) {
+            throw new IllegalArgumentException(x.getMessage(), x);
+        }
+    }
+    return uri;
+});
+config.addUriTranlator(uri -> {
+    if (true
+        && "https".equalsIgnoreCase(uri.getScheme()
+        && "schemas.acme.org".equalsIgnoreCase(uri.getHost())
+        && (-1 == uri.getPort() || 443 == uri.getPort())
+    ) {
+        try {
+            return new URI(
+                "http",
+                uri.getUserInfo(),
+                "test-schemas.acme.org",
+                8080,
+                uri.getPath(),
+                uri.getQuery(),
+                uri.getFragment()
+            );
+        } catch (URISyntaxException x) {
+            throw new IllegalArgumentException(x.getMessage(), x);
+        }
+    }
+    return uri;
+});
+config.addUriTranlator(URITranslator.map("http://test-schemas.acme.org:8080/Foo", "classpath://Foo");
+```
+
+Given a starting URI of `https://schemas.acme.org/Foo`, the configuration above produces the following translations (in order):
+
+1. The translation from HTTP to HTTPS does not occur since the original URI already specifies HTTPS.
+2. The second rule receives the original URI since nothing happened in the first rule. The second rule translates the URI from `https://schemas.acme.org/Foo` to `http://test-schemas.acme.org:8080/Foo` since the scheme, host and port match this rule.
+3. The third rule receives the URI produced by the second rule and performs a simple mapping to a local resource.
+
+Since all `JsonSchemaFactory`s are created from an optional `SchemaValidatorsConfig`, any `URITranslator`s added to the factory are evaluated after those provided by `SchemaValidatorsConfig`.
+
+### Deprecated
+
+Previously, this library supported simple mappings from one URI to another through `SchemaValidatorsConfig.setUriMappings()` and `JsonSchemaFactory.addUriMappings()`. Usage of these methods are still supported but are now discouraged. `URITranslator` provides a more powerful mechanism of dealing with URI mapping than what was provided before.
 
-please take a look at the test cases and the [PR](https://github.com/networknt/json-schema-validator/pull/274).

src/main/java/com/networknt/schema/JsonSchemaFactory.java

@@ -20,6 +20,7 @@
 import com.fasterxml.jackson.databind.ObjectMapper;
 import com.fasterxml.jackson.dataformat.yaml.YAMLMapper;
 import com.networknt.schema.uri.*;
+import com.networknt.schema.uri.URITranslator.CompositeURITranslator;
 import com.networknt.schema.urn.URNFactory;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -52,6 +53,7 @@ public static class Builder {
         private boolean forceHttps = true;
         private boolean removeEmptyFragmentSuffix = true;
         private boolean enableUriSchemaCache = true;
+        private final CompositeURITranslator uriTranslators = new CompositeURITranslator();
 
         public Builder() {
             // Adds support for creating {@link URL}s.
@@ -138,11 +140,22 @@ public Builder addMetaSchemas(final Collection<? extends JsonMetaSchema> jsonMet
             return this;
         }
 
+        /**
+         * @deprecated Use {@code addUriTranslator} instead.
+         */
+        @Deprecated
         public Builder addUriMappings(final Map<String, String> map) {
             this.uriMap.putAll(map);
             return this;
         }
 
+        public Builder addUriTranslator(URITranslator translator) {
+            if (null != translator) {
+                this.uriTranslators.add(translator);
+            }
+            return this;
+        }
+
         public Builder addUrnFactory(URNFactory urnFactory) {
             this.urnFactory = urnFactory;
             return this;
@@ -176,7 +189,8 @@ public JsonSchemaFactory build() {
                     uriMap,
                     forceHttps,
                     removeEmptyFragmentSuffix,
-                    enableUriSchemaCache
+                    enableUriSchemaCache,
+                    uriTranslators
             );
         }
     }
@@ -186,6 +200,7 @@ public JsonSchemaFactory build() {
     private final String defaultMetaSchemaURI;
     private final URISchemeFactory uriFactory;
     private final URISchemeFetcher uriFetcher;
+    private final CompositeURITranslator uriTranslators;
     private final URNFactory urnFactory;
     private final Map<String, JsonMetaSchema> jsonMetaSchemas;
     private final Map<String, String> uriMap;
@@ -206,7 +221,8 @@ private JsonSchemaFactory(
             final Map<String, String> uriMap,
             final boolean forceHttps,
             final boolean removeEmptyFragmentSuffix,
-            final boolean enableUriSchemaCache) {
+            final boolean enableUriSchemaCache,
+            final CompositeURITranslator uriTranslators) {
         if (jsonMapper == null) {
             throw new IllegalArgumentException("ObjectMapper must not be null");
         } else if (yamlMapper == null) {
@@ -223,6 +239,8 @@ private JsonSchemaFactory(
             throw new IllegalArgumentException("Meta Schema for default Meta Schema URI must be provided");
         } else if (uriMap == null) {
             throw new IllegalArgumentException("URL Mappings must not be null");
+        } else if (uriTranslators == null) {
+            throw new IllegalArgumentException("URI Translators must not be null");
         }
         this.jsonMapper = jsonMapper;
         this.yamlMapper = yamlMapper;
@@ -235,6 +253,7 @@ private JsonSchemaFactory(
         this.forceHttps = forceHttps;
         this.removeEmptyFragmentSuffix = removeEmptyFragmentSuffix;
         this.enableUriSchemaCache = enableUriSchemaCache;
+        this.uriTranslators = uriTranslators;
     }
 
     /**
@@ -301,6 +320,9 @@ public static Builder builder(final JsonSchemaFactory blueprint) {
                 .yamlMapper(blueprint.yamlMapper)
                 .addUriMappings(blueprint.uriMap);
 
+        for (URITranslator translator: blueprint.uriTranslators) {
+            builder = builder.addUriTranslator(translator);
+        }
         for (Map.Entry<String, URIFactory> entry : blueprint.uriFactory.getURIFactories().entrySet()) {
             builder = builder.uriFactory(entry.getValue(), entry.getKey());
         }
@@ -341,6 +363,10 @@ public URIFactory getUriFactory() {
         return this.uriFactory;
     }
 
+    public URITranslator getUriTranslator() {
+        return this.uriTranslators.with(URITranslator.map(uriMap));
+    }
+
     public JsonSchema getSchema(final String schema, final SchemaValidatorsConfig config) {
         try {
             final JsonNode schemaNode = jsonMapper.readTree(schema);
@@ -372,12 +398,11 @@ public JsonSchema getSchema(final InputStream schemaStream) {
     public JsonSchema getSchema(final URI schemaUri, final SchemaValidatorsConfig config) {
         try {
             InputStream inputStream = null;
-            final Map<String, String> map = (config != null) ? config.getUriMappings() : new HashMap<String, String>();
-            map.putAll(uriMap);
+            final URITranslator uriTranslator = null == config ? getUriTranslator() : config.getUriTranslator().with(getUriTranslator());
 
             final URI mappedUri;
             try {
-                mappedUri = this.uriFactory.create(map.get(schemaUri.toString()) != null ? map.get(schemaUri.toString()) : schemaUri.toString());
+                mappedUri = this.uriFactory.create(uriTranslator.translate(schemaUri).toString());
             } catch (IllegalArgumentException e) {
                 logger.error("Failed to create URI.", e);
                 throw new JsonSchemaException(e);

src/main/java/com/networknt/schema/SchemaValidatorsConfig.java

@@ -23,6 +23,8 @@
 import java.util.Set;
 
 import com.fasterxml.jackson.databind.JsonNode;
+import com.networknt.schema.uri.URITranslator;
+import com.networknt.schema.uri.URITranslator.CompositeURITranslator;
 import com.networknt.schema.walk.JsonSchemaWalkListener;
 
 public class SchemaValidatorsConfig {
@@ -70,6 +72,8 @@ public class SchemaValidatorsConfig {
      */
     private Map<String, String> uriMappings = new HashMap<String, String>();
 
+    private CompositeURITranslator uriTranslators = new CompositeURITranslator();
+
     /**
      * When a field is set as nullable in the OpenAPI specification, the schema validator validates that it is nullable
      * however continues with validation against the nullable field
@@ -130,11 +134,30 @@ public ApplyDefaultsStrategy getApplyDefaultsStrategy() {
         return applyDefaultsStrategy;
     }
 
+    public CompositeURITranslator getUriTranslator() {
+        return this.uriTranslators
+            .with(URITranslator.map(this.uriMappings));
+    }
+
+    public void addUriTranslator(URITranslator uriTranslator) {
+        if (null != uriTranslator) {
+            this.uriTranslators.add(uriTranslator);
+        }
+    }
+
+    /**
+     * @deprecated Use {@code getUriTranslator()} instead
+     */
+    @Deprecated
     public Map<String, String> getUriMappings() {
         // return a copy of the mappings
         return new HashMap<String, String>(uriMappings);
     }
 
+    /**
+     * @deprecated Use {@code addUriTranslator()} instead
+     */
+    @Deprecated
     public void setUriMappings(Map<String, String> uriMappings) {
         this.uriMappings = uriMappings;
     }