Draft pull request for 1.5 release (#331)

* Bump spring.version in /aws-serverless-java-container-spring (#319)

Bumps `spring.version` from 5.1.9.RELEASE to 5.2.3.RELEASE.

Updates `spring-webmvc` from 5.1.9.RELEASE to 5.2.3.RELEASE
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](spring-projects/spring-framework@v5.1.9.RELEASE...v5.2.3.RELEASE)

Updates `spring-test` from 5.1.9.RELEASE to 5.2.3.RELEASE
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](spring-projects/spring-framework@v5.1.9.RELEASE...v5.2.3.RELEASE)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump spring-webflux in /aws-serverless-java-container-springboot2 (#318)

Bumps [spring-webflux](https://github.com/spring-projects/spring-framework) from 5.1.9.RELEASE to 5.2.0.RELEASE.
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](spring-projects/spring-framework@v5.1.9.RELEASE...v5.2.0.RELEASE)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* ci: Fixing Spring build to use 5.2 as latest

* chore(deps): Bump Spring 5.1 path release to address a security vulnerability

* chore(deps): Fixing usual spring dependency mess with exlusions out of the spring-security package used in the tests

* Fix for issue #317 (#323)

* fix issue 317 - use charset from request

* update dependencies

* update build dependencies, remove spring boot 2.0.x

* restoring ci config

Co-authored-by: Stefano Buliani <[email protected]>

* test: Fixed Spring security tests for SpringBoot 2, added validation tests and updated servlet tests to use the new servletApplication option

* fix: Avoid flushing the response buffer if we are dispatching the request asynchronously. This was causing race conditions in the SpringBoot 2 WebFlux implementation - requests that had to run through security or validation filters took longer and the library flushed an empty request, which caused the status code to default to 200. This fix addresses issues #279, #304, and #306

* chore(deps): Bump spring dependency version and added webmvc optional dependency to truly support Servlet-only server

* feat: New application type parameter to SpringBootLambdaContainerHandler that tells the framework whether to start a reactive or servlet-based embedded server. Also added a new servletApplication method to the builder object.

* test: Fixed UTF-8 encoding test

* ci: Fixed dependencies for CI run on SpringBoot 2

* ci: More Spring dependency convergence issues during CI

* fix: Added null-check on getServerName in case the multi-value headers property is null. Unlikely outside of tests but better safe than sorry. This addresses #327

* fix: Changed servlet initialization mechanism so that servlet that requests load on startup are initialized right away, as part of the initialization() method call in LambdaServletContainerHandler. Also centralized the lazy Servlet initialization to the ServletExecutionFilter so that we don't have code scattered all around. This begins to address #287

* feat: Added new 0-parameter constructor for async initializer that uses the actual JVM start time to calculate the timeout milliseconds. Also added the new method to the builder object and deprecated the current method that receives a milliseconds epoch parameter. I'm not deprecating the constructor of the async initializer class that receives the parameter as it may still be useful for tests. This change was suggested in #287

* fix: Updated SpringBoot 1.x handler to use the new servlet initialization mechanism

* ci: switch SpringBoot slow integration test to use a custom async time since the JVM is reused for both tests in the and we cannot reuse the actual JVM init time

* feat: New models for HTTP API support for #329

* feat: First implementation of HTTP API servlet request, request reader, and security context writer - continuing to address #329

* test: Basic unit tests for the new HTTP API support in core library (#329)

* feat: Updated log formatter to support both versions (1 and 2) of the proxy request model (#329)

* feat: Further generified request readers to read to a generic HttpServletRequest rather than specific implementations of it. This makes it easier to create container handler implementations that support HTTP API, API Gateway, and ALB (#329)

* test: Fixed tests for new logged and generified request readers

* feat: Added HTTP API support to Jersey implementation with new getHttpApiV2ProxyHandler method (#329)

* feat: Added HTTP API support to Spark implementation (#329)

* feat: Added HTTP API support to Spring implementation (#329)

* feat: HTTP API support in SpringBoot 2 implementation. bug: Fixed an issue with the implementation of AsyncContext where it wasn't dispatching if the handler wasn't set

* feat: First pass of HTTP API support in struts 2 implementation (#329)

* fix: Added support for HTTP APIs to the request dispatcher

* chore(deps): Dependency bump all around. Rotated Jersey ci versions

* fix: Updated stream handling logic to work with reactive applications as suggested in #316

* test: Added unit test to replicate #333

* feat: New configuration parameter to skip exception mapping and allow exception to bubble up from #307

* fix: Fixed spotbugs issue in RuntimeException cast

* test: Added tests for more complex content types mentioned in issue #315

* docs: Updated samples to support SAM CLI operations out of the box to address #293 and switched to HTTP API by default

* feat: Updated archetypes to work out of the box with the SAM CLI, continuing to address #293

* chore: License header pass on the entire project

* fix: Set default value for setDisableException mapper in config to false

* fix: Updated default initialization timeout to 20 seconds

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Eran Medan <[email protected]>

Author: 3 people

.github/workflows/continuous-integration-workflow.yml

@@ -22,12 +22,12 @@ jobs:
       - uses: actions/checkout@v2
       - name: Build latest
         run: ./gha_build.sh jersey true true
-      - name: Build Jersey 2.26
-        run: ./gha_build.sh jersey false false -Djersey.version=2.26
       - name: Build Jersey 2.27
         run: ./gha_build.sh jersey false false -Djersey.version=2.27
       - name: Build Jersey 2.28
         run: ./gha_build.sh jersey false false -Djersey.version=2.28
+      - name: Build Jersey 2.29
+        run: ./gha_build.sh jersey false false -Djersey.version=2.29.1
 
   build_spark:
     name: Build and test Spark
@@ -45,17 +45,15 @@ jobs:
     steps:
       - uses: actions/checkout@v2
       - name: Build latest
-        run: ./gha_build.sh spring true true
+        # we reduce the minCoverage for this run because it will skip the SpringBoot 1.5 tests since they are no longer compatible with
+        # Spring core 5.2 and above. SpringBoot 1.5 is deprecated
+        run: ./gha_build.sh spring true true -Djacoco.minCoverage=0.4
       - name: Build Spring 4.3
         run: ./gha_build.sh spring false false -Dspring.version=4.3.25.RELEASE -Dspring-security.version=4.2.13.RELEASE
       - name: Build Spring 5.0
-        run: ./gha_build.sh spring false false -Dspring.version=5.0.15.RELEASE -Dspring-security.version=5.0.13.RELEASE
+        run: ./gha_build.sh spring false false -Dspring.version=5.0.16.RELEASE -Dspring-security.version=5.0.14.RELEASE
       - name: Build Spring 5.1
         run: ./gha_build.sh spring false false -Dspring.version=5.1.14.RELEASE -Dspring-security.version=5.1.8.RELEASE
-      - name: Build Spring 5.2
-        # we reduce the minCoverage for this run because it will skip the SpringBoot 1.5 tests since they are no longer compatible with
-        # Spring core 5.2 and above. SpringBoot 1.5 is deprecated
-        run: ./gha_build.sh spring false false -Dspring.version=5.2.5.RELEASE -Dspring-security.version=5.2.2.RELEASE -Djacoco.minCoverage=0.4
 
   build_springboot2:
     name: Build and test SpringBoot 2
@@ -65,11 +63,9 @@ jobs:
       - name: Build latest
         run: ./gha_build.sh springboot2 true true
       - name: Build Spring Boot 2.0
-        run: ./gha_build.sh springboot2 false false -Dspringboot.version=2.0.9.RELEASE -Dspring.version=5.0.13.RELEASE -Dspringsecurity.version=5.0.12.RELEASE
+        run: ./gha_build.sh springboot2 false false -Dspringboot.version=2.0.9.RELEASE -Dspring.version=5.0.16.RELEASE -Dspringsecurity.version=5.0.14.RELEASE
       - name: Build Spring Boot 2.1
-        run: ./gha_build.sh springboot2 false false -Dspringboot.version=2.1.10.RELEASE -Dspring.version=5.1.11.RELEASE -Dspringsecurity.version=5.1.7.RELEASE
-      - name: Build Spring Boot 2.2
-        run: ./gha_build.sh springboot2 false false -Dspringboot.version=2.2.1.RELEASE -Dspring.version=5.2.1.RELEASE -Dspringsecurity.version=5.2.1.RELEASE
+        run: ./gha_build.sh springboot2 false false -Dspringboot.version=2.1.12.RELEASE -Dspring.version=5.1.13.RELEASE -Dspringsecurity.version=5.1.8.RELEASE
 
   build_struts2:
     name: Build and test Struts 2

.gitignore

@@ -29,3 +29,7 @@ gradlew*
 
 # Exclude maven wrapper
 !/.mvn/wrapper/maven-wrapper.jar
+
+# SAM files
+samconfig.toml
+.aws-sam/

aws-serverless-java-container-core/src/main/java/com/amazonaws/serverless/proxy/AsyncInitializationWrapper.java

@@ -1,3 +1,15 @@
+/*
+ * Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance
+ * with the License. A copy of the License is located at
+ *
+ * http://aws.amazon.com/apache2.0/
+ *
+ * or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES
+ * OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions
+ * and limitations under the License.
+ */
 package com.amazonaws.serverless.proxy;
 
 import com.amazonaws.serverless.exceptions.ContainerInitializationException;
@@ -7,6 +19,7 @@
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
+import java.lang.management.ManagementFactory;
 import java.time.Instant;
 import java.util.concurrent.CountDownLatch;
 import java.util.concurrent.TimeUnit;
@@ -25,7 +38,7 @@
  * seconds has already been used up.
  */
 public class AsyncInitializationWrapper extends InitializationWrapper {
-    private static final int INIT_GRACE_TIME_MS = 250;
+    private int INIT_GRACE_TIME_MS = 250;
     private static final int LAMBDA_MAX_INIT_TIME_MS = 10_000;
 
     private CountDownLatch initializationLatch;
@@ -41,6 +54,15 @@ public AsyncInitializationWrapper(long startTime) {
         actualStartTime = startTime;
     }
 
+    /**
+     * Creates a new instance of the async initializer using the actual JVM start time as the starting point to measure
+     * the 10 seconds timeout.
+     */
+    public AsyncInitializationWrapper() {
+        actualStartTime = ManagementFactory.getRuntimeMXBean().getStartTime();
+        INIT_GRACE_TIME_MS = 150;
+    }
+
     @Override
     public void start(LambdaContainerHandler handler) throws ContainerInitializationException {
         initializationLatch = new CountDownLatch(1);
@@ -50,7 +72,7 @@ public void start(LambdaContainerHandler handler) throws ContainerInitialization
         try {
             long curTime = Instant.now().toEpochMilli();
             // account for the time it took to call the various constructors with the actual start time + a grace of 500ms
-            long awaitTime = LAMBDA_MAX_INIT_TIME_MS - (curTime - actualStartTime) - INIT_GRACE_TIME_MS;
+            long awaitTime = (actualStartTime + LAMBDA_MAX_INIT_TIME_MS) - curTime - INIT_GRACE_TIME_MS;
             log.info("Async initialization will wait for " + awaitTime + "ms");
             if (!initializationLatch.await(awaitTime, TimeUnit.MILLISECONDS)) {
                 log.info("Initialization took longer than " + LAMBDA_MAX_INIT_TIME_MS + ", setting new CountDownLatch and " +
@@ -65,6 +87,10 @@ public void start(LambdaContainerHandler handler) throws ContainerInitialization
         }
     }
 
+    public long getActualStartTimeMs() {
+        return actualStartTime;
+    }
+
     @Override
     public CountDownLatch getInitializationLatch() {
         return initializationLatch;

aws-serverless-java-container-core/src/main/java/com/amazonaws/serverless/proxy/AwsHttpApiV2SecurityContextWriter.java

@@ -0,0 +1,26 @@
+/*
+ * Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance
+ * with the License. A copy of the License is located at
+ *
+ * http://aws.amazon.com/apache2.0/
+ *
+ * or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES
+ * OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions
+ * and limitations under the License.
+ */
+package com.amazonaws.serverless.proxy;
+
+import com.amazonaws.serverless.proxy.internal.jaxrs.AwsHttpApiV2SecurityContext;
+import com.amazonaws.serverless.proxy.model.HttpApiV2ProxyRequest;
+import com.amazonaws.services.lambda.runtime.Context;
+
+import javax.ws.rs.core.SecurityContext;
+
+public class AwsHttpApiV2SecurityContextWriter implements SecurityContextWriter<HttpApiV2ProxyRequest> {
+    @Override
+    public SecurityContext writeSecurityContext(HttpApiV2ProxyRequest event, Context lambdaContext) {
+        return new AwsHttpApiV2SecurityContext(lambdaContext, event);
+    }
+}

aws-serverless-java-container-core/src/main/java/com/amazonaws/serverless/proxy/InitializationWrapper.java

@@ -1,3 +1,15 @@
+/*
+ * Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance
+ * with the License. A copy of the License is located at
+ *
+ * http://aws.amazon.com/apache2.0/
+ *
+ * or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES
+ * OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions
+ * and limitations under the License.
+ */
 package com.amazonaws.serverless.proxy;
 
 import com.amazonaws.serverless.exceptions.ContainerInitializationException;

aws-serverless-java-container-core/src/main/java/com/amazonaws/serverless/proxy/LogFormatter.java

@@ -1,9 +1,19 @@
+/*
+ * Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance
+ * with the License. A copy of the License is located at
+ *
+ * http://aws.amazon.com/apache2.0/
+ *
+ * or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES
+ * OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions
+ * and limitations under the License.
+ */
 package com.amazonaws.serverless.proxy;
 
-
 import javax.ws.rs.core.SecurityContext;
 
-
 /**
  * Implementations of the log formatter interface are used by {@link com.amazonaws.serverless.proxy.internal.LambdaContainerHandler} class to log each request
  * processed in the container. You can set the log formatter using the {@link com.amazonaws.serverless.proxy.internal.LambdaContainerHandler#setLogFormatter(LogFormatter)}

aws-serverless-java-container-core/src/main/java/com/amazonaws/serverless/proxy/RequestReader.java

@@ -65,7 +65,20 @@ public abstract class RequestReader<RequestType, ContainerRequestType> {
      */
     public static final String JAX_SECURITY_CONTEXT_PROPERTY = "com.amazonaws.serverless.jaxrs.securityContext";
 
+    /**
+     * The key for the <strong>HTTP API</strong> request context passed by the services
+     */
+    public static final String HTTP_API_CONTEXT_PROPERTY = "com.amazonaws.httpapi.request.context";
 
+    /**
+     * The key for the <strong>HTTP API</strong> stage variables
+     */
+    public static final String HTTP_API_STAGE_VARS_PROPERTY = "com.amazonaws.httpapi.stage.variables";
+
+    /**
+     * The key for the <strong>HTTP API</strong> proxy request event
+     */
+    public static final String HTTP_API_EVENT_PROPERTY = "com.amazonaws.httpapi.request";
 
     //-------------------------------------------------------------
     // Methods - Abstract

aws-serverless-java-container-core/src/main/java/com/amazonaws/serverless/proxy/internal/LambdaContainerHandler.java

@@ -223,7 +223,15 @@ public ResponseType proxy(RequestType request, Context context) {
             // the latch will do nothing
             latch.countDown();
 
-            return exceptionHandler.handle(e);
+            if (getContainerConfig().isDisableExceptionMapper()) {
+                if (e instanceof RuntimeException) {
+                    throw (RuntimeException) e;
+                } else {
+                    throw new RuntimeException(e);
+                }
+            } else {
+                return exceptionHandler.handle(e);
+            }
         }
     }
 

aws-serverless-java-container-core/src/main/java/com/amazonaws/serverless/proxy/internal/SecurityUtils.java

@@ -1,6 +1,17 @@
+/*
+ * Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance
+ * with the License. A copy of the License is located at
+ *
+ * http://aws.amazon.com/apache2.0/
+ *
+ * or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES
+ * OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions
+ * and limitations under the License.
+ */
 package com.amazonaws.serverless.proxy.internal;
 
-
 import edu.umd.cs.findbugs.annotations.SuppressFBWarnings;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -11,7 +22,6 @@
 import java.util.Locale;
 import java.util.Set;
 
-
 /**
  * This class contains utility methods to address FSB security issues found in the application, such as string sanitization
  * and file path validation.

aws-serverless-java-container-core/src/main/java/com/amazonaws/serverless/proxy/internal/jaxrs/AwsHttpApiV2SecurityContext.java

@@ -0,0 +1,102 @@
+/*
+ * Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance
+ * with the License. A copy of the License is located at
+ *
+ * http://aws.amazon.com/apache2.0/
+ *
+ * or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES
+ * OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions
+ * and limitations under the License.
+ */
+package com.amazonaws.serverless.proxy.internal.jaxrs;
+
+import com.amazonaws.serverless.proxy.internal.LambdaContainerHandler;
+import com.amazonaws.serverless.proxy.internal.SecurityUtils;
+import com.amazonaws.serverless.proxy.model.HttpApiV2ProxyRequest;
+import com.amazonaws.services.lambda.runtime.Context;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.JsonNode;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.ws.rs.core.HttpHeaders;
+import javax.ws.rs.core.SecurityContext;
+import java.nio.charset.StandardCharsets;
+import java.security.Principal;
+import java.util.Base64;
+
+public class AwsHttpApiV2SecurityContext implements SecurityContext {
+    public static final String AUTH_SCHEME_JWT = "JWT";
+
+    private static Logger log = LoggerFactory.getLogger(AwsHttpApiV2SecurityContext.class);
+
+    private Context lambdaContext;
+    private HttpApiV2ProxyRequest event;
+
+    public AwsHttpApiV2SecurityContext(final Context lambdaCtx, final HttpApiV2ProxyRequest request) {
+        lambdaContext = lambdaCtx;
+        event = request;
+    }
+
+    @Override
+    public Principal getUserPrincipal() {
+        if (getAuthenticationScheme() == null || !event.getHeaders().containsKey(HttpHeaders.AUTHORIZATION)) {
+            return null;
+        }
+
+        String authValue = event.getHeaders().get(HttpHeaders.AUTHORIZATION);
+        if (authValue.startsWith("Bearer ")) {
+            authValue = authValue.replace("Bearer ", "");
+        }
+        String[] parts = authValue.split("\\.");
+        if (parts.length != 3) {
+            log.warn("Could not parse JWT token for requestId: " + SecurityUtils.crlf(event.getRequestContext().getRequestId()));
+            return null;
+        }
+        String decodedBody = new String(Base64.getMimeDecoder().decode(parts[1]), StandardCharsets.UTF_8);
+        try {
+            JsonNode parsedBody = LambdaContainerHandler.getObjectMapper().readTree(decodedBody);
+            if (!parsedBody.isObject() && parsedBody.has("sub")) {
+                log.debug("Could not find \"sub\" field in JWT body for requestId: " + SecurityUtils.crlf(event.getRequestContext().getRequestId()));
+                return null;
+            }
+            String subject = parsedBody.get("sub").asText();
+            return (() -> {
+                return subject;
+            });
+        } catch (JsonProcessingException e) {
+            log.error("Error while attempting to parse JWT body for requestId: " + SecurityUtils.crlf(event.getRequestContext().getRequestId()), e);
+            return null;
+        }
+
+    }
+
+    @Override
+    public boolean isUserInRole(String s) {
+        if (getAuthenticationScheme() == null) {
+            return false;
+        }
+
+        return event.getRequestContext().getAuthorizer().getJwtAuthorizer().getScopes().contains(s) ||
+                event.getRequestContext().getAuthorizer().getJwtAuthorizer().getClaims().containsKey(s);
+
+    }
+
+    @Override
+    public boolean isSecure() {
+        return getAuthenticationScheme() != null;
+    }
+
+    @Override
+    public String getAuthenticationScheme() {
+        if (event.getRequestContext().getAuthorizer() == null) {
+            return null;
+        }
+        if (event.getRequestContext().getAuthorizer().isJwt()) {
+            return AUTH_SCHEME_JWT;
+        }
+        return null;
+    }
+}