Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion src/main/java/io/vertx/openapi/contract/MediaType.java
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,10 @@ public interface MediaType extends OpenAPIObject {
String APPLICATION_HAL_JSON = "application/hal+json";
String APPLICATION_JSON = HttpHeaderValues.APPLICATION_JSON.toString();
String APPLICATION_JSON_UTF8 = APPLICATION_JSON + "; charset=utf-8";
String APPLICATION_OCTET_STREAM = HttpHeaderValues.APPLICATION_OCTET_STREAM.toString();
String MULTIPART_FORM_DATA = HttpHeaderValues.MULTIPART_FORM_DATA.toString();
List<String> SUPPORTED_MEDIA_TYPES = Arrays.asList(APPLICATION_JSON, APPLICATION_JSON_UTF8, MULTIPART_FORM_DATA, APPLICATION_HAL_JSON);
List<String> SUPPORTED_MEDIA_TYPES = Arrays.asList(APPLICATION_JSON, APPLICATION_JSON_UTF8,
MULTIPART_FORM_DATA, APPLICATION_HAL_JSON, APPLICATION_OCTET_STREAM);

static boolean isMediaTypeSupported(String type) {
return SUPPORTED_MEDIA_TYPES.contains(type.toLowerCase());
Expand Down
18 changes: 17 additions & 1 deletion src/main/java/io/vertx/openapi/contract/impl/MediaTypeImpl.java
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,23 @@ public MediaTypeImpl(String identifier, JsonObject mediaTypeModel) {
this.mediaTypeModel = mediaTypeModel;
JsonObject schemaJson = mediaTypeModel.getJsonObject(KEY_SCHEMA);
if (schemaJson == null || schemaJson.isEmpty()) {
throw createUnsupportedFeature("Media Type without a schema");
// Inject default value if schema is left out
// by using shortcut "application/octet-stream: {}"
// See https://www.openapis.org/blog/2021/02/16/migrating-from-openapi-3-0-to-3-1-0
if (identifier.equalsIgnoreCase(MediaType.APPLICATION_OCTET_STREAM)) {
// In OpenAPI v3.0, describing file uploads is signalled with a type:
// string and the format set to byte, binary, or base64.
// In OpenAPI v3.1, JSON Schema helps make this far more clear with
// its contentEncoding and contentMediaType keywords,
// which are designed for exactly this sort of use.
// See https://datatracker.ietf.org/doc/html/draft-bhutton-json-schema-validation-00#section-8.6
schemaJson = new JsonObject()
.put("type", "string")
.put("format", "binary")
.put("contentMediaType", "application/octet-stream");
Comment on lines +43 to +46
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should become a constant maybe in MediaType. I would also remove contentMediaType as in the example only format and type is mentioned. This is then also compatible with 3.0

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll replace it as a constant in MediaType.

But stripping the contentMediaType isn't a good idea, because it replaces format in OpenAPI 3.1. I've intentionally placed both keys in that JSON object to be compatible with OpenAPI 3.x.

That way it's future proof, because later adaptions of this library for OpenAPI 3.1 or later might break.

Am I overthinking that part?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mh maybe you are right, but I think we can simply ignore contentMediaType , because the MediaType is already specified with if (identifier.equalsIgnoreCase(MediaType.APPLICATION_OCTET_STREAM)) {

JSON Schema also offers a contentMediaType keyword. However, when the media type is already specified by the Media Type Object's key, or by the contentType field of an Encoding Object, the contentMediaType keyword SHALL be ignored if present.

Would you agree?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should become a constant maybe in MediaType. I would also remove contentMediaType as in the example only format and type is mentioned. This is then also compatible with 3.0

If someone uses the alias application/octet-stream: {}, it's safe to assume that the contract file is of version 3.1, because that wasn't allowed before in 3.0 and lower. Therefore I'm removing format from the schemaJson.
The final schema object that gets injected into the media type object would look like the following:

new JsonObject()
          .put("type", "string")
          .put("contentMediaType", "application/octet-stream");

Mh maybe you are right, but I think we can simply ignore contentMediaType , because the MediaType is already specified with if (identifier.equalsIgnoreCase(MediaType.APPLICATION_OCTET_STREAM)) {

Yeah, that makes sense to ignore it here.

Summary: For contract validation we place a default schema object like seen above, if someone uses the alias definition introduced in OpenAPI 3.1. For request validation we simplify the validation if the media type object ID is application/octet-stream.

Sounds good?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the benefit of having contentMediaType in the default schema object? I still don't understand it.

Why can't we simply use

new JsonObject()
          .put("type", "string")
          .put("format", "binary")

?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default schema object must only be applied if someone has an openapi-contract.yml file with OpenAPI 3.1. Before that, no shortcuts like {} could be used in the schema file.

In OpenAPI 3.0 you must write:

# an arbitrary binary file:
content:
    application/octet-stream:
        schema:
            type: string
            format: binary

In OpenAPI 3.1 you (can) now write:

# a PNG image as a binary file:
content:
    image/png: {}
# an arbitrary binary file:
content:
    application/octet-stream: {}

And that gets expanded to:

# a PNG image as a binary file:
content:
    image/png:
        schema:
            type: string
            contentMediaType: image/png
            contentEncoding: base64
# an arbitrary binary file:
content:
    application/octet-stream:
        schema:
            type: string
            contentMediaType: application/octet-stream

It really is confusing! As of my understanding, in OpenAPI 3.1, format has been replaced by contentMediaType when dealing with Schema Objects in relation to binary file uploads. See https://spec.openapis.org/oas/latest.html#considerations-for-file-uploads

The task of this library must be the expansion before the validation of the contract. Right now I'm only providing that expansion for application/octet-stream, because everything else isn't part of this PR.

We could still use your approach of the old way of declaring it, but maybe that will create confusion in later stages of development of this library? Supporting multiple versions of OpenAPI is challenging.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Later in the code, if necessary, you can distinguish between 3.0 and 3.1 only be checking the keys of the schema object. If format is missing but contentMediaType is present, it must be 3.1. Otherwise it will be 3.0 schema. This might be come in handy...

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I don't get the specs .... maybe we don't need a schema object at all. It seems like it is intentionally left out when using {} in the contract.

Then it is treated like

Content transferred in binary (octet-stream)

OK, so leave out the schema object? What do you think?

} else {
throw createUnsupportedFeature("Media Type without a schema");
}
}
schema = JsonSchema.of(schemaJson);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
import io.vertx.openapi.contract.OpenAPIContract;
import io.vertx.openapi.contract.Operation;
import io.vertx.openapi.validation.transformer.ApplicationJsonTransformer;
import io.vertx.openapi.validation.transformer.ApplicationOctetStreamTransformer;
import io.vertx.openapi.validation.transformer.BodyTransformer;
import io.vertx.openapi.validation.transformer.MultipartFormTransformer;

Expand All @@ -42,6 +43,7 @@ public BaseValidator(Vertx vertx, OpenAPIContract contract) {
bodyTransformers.put(MediaType.APPLICATION_JSON_UTF8, new ApplicationJsonTransformer());
bodyTransformers.put(MediaType.MULTIPART_FORM_DATA, new MultipartFormTransformer());
bodyTransformers.put(MediaType.APPLICATION_HAL_JSON, new ApplicationJsonTransformer());
bodyTransformers.put(MediaType.APPLICATION_OCTET_STREAM, new ApplicationOctetStreamTransformer());
}

// VisibleForTesting
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
import io.vertx.core.Future;
import io.vertx.core.Vertx;
import io.vertx.core.http.HttpServerRequest;
import io.vertx.json.schema.JsonSchema;
import io.vertx.json.schema.JsonSchemaValidationException;
import io.vertx.json.schema.OutputUnit;
import io.vertx.openapi.contract.MediaType;
Expand Down Expand Up @@ -153,6 +154,28 @@ RequestParameter validateBody(RequestBody requestBody, ValidatableRequest reques
throw new ValidatorException("The format of the request body is not supported", UNSUPPORTED_VALUE_FORMAT);
}
Object transformedValue = transformer.transformRequest(mediaType, request);
// Corner case for "application/octet-stream" media type:
// Skip full schema validation if the following simplified validation succeeds,
// otherwise validate with JSON Validator and fail the validation
// of the transformed value eventually.
if (mediaType.getIdentifier().equals(MediaType.APPLICATION_OCTET_STREAM)) {
JsonSchema schema = mediaType.getSchema();
String schemaTypeValue = schema.get("type", null);
if (schemaTypeValue != null && schemaTypeValue.equals("string")) {
String schemaFormatValue = schema.get("format", null);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for format it's the same as for type. If there is a chance that this required information is missing, it should fail when building the contract object, not when we do a validation.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

format can be null, since OpenAPI 3.1 replaced it with contentMediaType. I'm merely providing a fallback if someone still uses format in the contract file.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The simplified validation procedure has to deal with both 3.0 and 3.1 of OpenAPI. Therefore the statements used seem to be redundant.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, you are right, format can be null. But as mentioned in the comment below, I'm not convinced by the current implementation of the "schema validation bypass".

My concerns:

  • RequestValidatorImpl is already a large class
  • different logic for different versions -> makes it complex
  • you have to take care for corner cases

But I understand the benefit of having a schema validation bypass, especially for binary data.

Maybe we can call a method like "canSkipValidation" and extract this method and its logic into a separated class. You could pass the contract reference to this method to check if the contract is 3.0 or 3.1 which makes the logic in this method less complex.

What do you think?

// Case for OpenAPI 3.0 and OpenAPI 3.1 (backwards compatibility)
if (schemaFormatValue != null && schemaFormatValue.equals("binary")) {
return new RequestParameterImpl(transformedValue);
} else {
// Case only for OpenAPI 3.1
String schemaContentMediaType = schema.get("contentMediaType", null);
if (schemaContentMediaType != null
&& schemaContentMediaType.equals(MediaType.APPLICATION_OCTET_STREAM)) {
return new RequestParameterImpl(transformedValue);
Comment on lines +167 to +174
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't agree to this. As long as the type is string, you could have properties like maxLength, couldn't you?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I don't understand that one?

The null check for conentMediaType is because of the support for OpenAPI 3.1. Someone could use a contract file for version 3.0 where that field is not a requirement.

The verification of the value of contentMediaType is necessary because it could have another value.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I mean is, that you skip the validation. Couldn't it be possible to define a schema like this?

type: string
contentMediaType: application/octet-stream
minLength: 2

}
}
}
}
OutputUnit result = contract.getSchemaRepository().validator(mediaType.getSchema()).validate(transformedValue);

try {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
/*
* Copyright (c) 2024, Lucimber UG
*
* This program and the accompanying materials are made available under the
* terms of the Eclipse Public License 2.0 which is available at
* http://www.eclipse.org/legal/epl-2.0, or the Apache License, Version 2.0
* which is available at https://www.apache.org/licenses/LICENSE-2.0.
*
* SPDX-License-Identifier: EPL-2.0 OR Apache-2.0
*
*/

package io.vertx.openapi.validation.transformer;

import io.vertx.core.buffer.Buffer;
import io.vertx.openapi.contract.MediaType;
import io.vertx.openapi.validation.ValidatableRequest;
import io.vertx.openapi.validation.ValidatableResponse;
import io.vertx.openapi.validation.ValidatorException;

import static io.vertx.openapi.contract.MediaType.APPLICATION_OCTET_STREAM;
import static io.vertx.openapi.validation.ValidatorErrorType.MISSING_REQUIRED_PARAMETER;

public class ApplicationOctetStreamTransformer implements BodyTransformer {

@Override
public Object transformRequest(MediaType type, ValidatableRequest request) {
return transform(type, request.getBody().getBuffer(), request.getContentType(), "request");
}

@Override
public Object transformResponse(MediaType type, ValidatableResponse response) {
return transform(type, response.getBody().getBuffer(), response.getContentType(), "response");
}

private Object transform(MediaType type, Buffer body, String contentType,
String responseOrRequest) {
if (contentType == null || contentType.isEmpty()
|| !contentType.equalsIgnoreCase(APPLICATION_OCTET_STREAM)) {
String msg = "The " + responseOrRequest
+ " doesn't contain the required content-type header application/octet-stream.";
throw new ValidatorException(msg, MISSING_REQUIRED_PARAMETER);
}
return body;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ private static Stream<Arguments> testExceptions() {
Arguments.of("0002_RequestBody_With_Content_Type_Text_Plain", UNSUPPORTED_FEATURE,
"The passed OpenAPI contract contains a feature that is not supported: Operation dummyOperation defines a " +
"request body with an unsupported media type. Supported: application/json, application/json; charset=utf-8," +
" multipart/form-data, application/hal+json")
" multipart/form-data, application/hal+json, application/octet-stream")
);
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ private static Stream<Arguments> testExceptions() {
Arguments.of("0000_Response_With_Content_Type_Text_Plain", UNSUPPORTED_FEATURE,
"The passed OpenAPI contract contains a feature that is not supported: Operation dummyOperation defines a " +
"response with an unsupported media type. Supported: application/json, application/json; charset=utf-8, " +
"multipart/form-data, application/hal+json")
"multipart/form-data, application/hal+json, application/octet-stream")
);
}

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
/*
* Copyright (c) 2024, Lucimber UG
*
* This program and the accompanying materials are made available under the
* terms of the Eclipse Public License 2.0 which is available at
* http://www.eclipse.org/legal/epl-2.0, or the Apache License, Version 2.0
* which is available at https://www.apache.org/licenses/LICENSE-2.0.
*
* SPDX-License-Identifier: EPL-2.0 OR Apache-2.0
*
*/

package io.vertx.openapi.validation.transformer;

import io.vertx.core.buffer.Buffer;
import io.vertx.openapi.validation.ValidatableRequest;
import io.vertx.openapi.validation.ValidatableResponse;
import io.vertx.openapi.validation.ValidatorException;
import org.junit.jupiter.api.Test;

import java.util.Random;

import static com.google.common.truth.Truth.assertThat;
import static io.netty.handler.codec.http.HttpHeaderValues.APPLICATION_JSON;
import static io.netty.handler.codec.http.HttpHeaderValues.APPLICATION_OCTET_STREAM;
import static io.vertx.openapi.MockHelper.mockValidatableRequest;
import static io.vertx.openapi.validation.ValidatorErrorType.MISSING_REQUIRED_PARAMETER;
import static org.junit.jupiter.api.Assertions.assertThrows;

class ApplicationOctetStreamTransformerTest {
private final BodyTransformer transformer = new ApplicationOctetStreamTransformer();
private final Random random = new Random();

@Test
void testTransformRequest() {
byte[] bytes = new byte[102400]; // Mimic body of 100 Kibibyte
random.nextBytes(bytes);
Buffer dummyBody = Buffer.buffer(bytes);
ValidatableRequest request = mockValidatableRequest(dummyBody, APPLICATION_OCTET_STREAM.toString());
assertThat(transformer.transformRequest(null, request)).isEqualTo(dummyBody);
}

@Test
void testTransformRequestThrows() {
ValidatorException exception =
assertThrows(ValidatorException.class, () -> transformer.transformRequest(null,
mockValidatableRequest(Buffer.buffer("\"foobar"), APPLICATION_JSON.toString())));
String expectedMsg = "The request doesn't contain" +
" the required content-type header application/octet-stream.";
assertThat(exception.type()).isEqualTo(MISSING_REQUIRED_PARAMETER);
assertThat(exception).hasMessageThat().isEqualTo(expectedMsg);
}

@Test
void testTransformResponse() {
byte[] bytes = new byte[102400]; // Mimic body of 100 Kibibyte
random.nextBytes(bytes);
Buffer dummyBody = Buffer.buffer(bytes);
ValidatableResponse response = ValidatableResponse.create(200, dummyBody,
APPLICATION_OCTET_STREAM.toString());
assertThat(transformer.transformResponse(null, response)).isEqualTo(dummyBody);
}

@Test
void testTransformResponseThrows() {
ValidatableResponse response = ValidatableResponse
.create(200, Buffer.buffer("\"foobar"), APPLICATION_JSON.toString());
ValidatorException exception =
assertThrows(ValidatorException.class, () -> transformer.transformResponse(null, response));
String expectedMsg = "The response doesn't contain" +
" the required content-type header application/octet-stream.";
assertThat(exception.type()).isEqualTo(MISSING_REQUIRED_PARAMETER);
assertThat(exception).hasMessageThat().isEqualTo(expectedMsg);
}
}