SecretsManagerException 1 validation error detected: Value null at 'secretId' failed to satisfy constraint: Member must not be null

[niqo01]

Describe the bug

Kotlin aws sdk is returning an error when running a Kotlin lambda fetching a secret value using a secret Arn from an event parameter.

Regression Issue

• Select this option if this issue appears to be a regression.

Expected behavior

No error, secret value is returned.

Current behavior

1 validation error detected: Value null at 'secretId' failed to satisfy constraint: Member must not be null: aws.sdk.kotlin.services.secretsmanager.model.SecretsManagerException
aws.sdk.kotlin.services.secretsmanager.model.SecretsManagerException: 1 validation error detected: Value null at 'secretId' failed to satisfy constraint: Member must not be null
	at aws.sdk.kotlin.services.secretsmanager.serde.GetSecretValueOperationDeserializerKt.throwGetSecretValueError(GetSecretValueOperationDeserializer.kt:67)
	at aws.sdk.kotlin.services.secretsmanager.serde.GetSecretValueOperationDeserializerKt.access$throwGetSecretValueError(GetSecretValueOperationDeserializer.kt:1)
	at aws.sdk.kotlin.services.secretsmanager.serde.GetSecretValueOperationDeserializer.deserialize(GetSecretValueOperationDeserializer.kt:37)
	at aws.sdk.kotlin.services.secretsmanager.serde.GetSecretValueOperationDeserializer.deserialize(GetSecretValueOperationDeserializer.kt:32)
	at aws.smithy.kotlin.runtime.http.operation.DeserializeHandler.call(SdkOperationExecution.kt:347)
	at aws.smithy.kotlin.runtime.http.operation.DeserializeHandler$call$1.invokeSuspend(SdkOperationExecution.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:101)
	at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:263)
	at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:95)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:69)
	at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:47)
	at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)

I enabled the SDK logging and got:

"loggerName": "httpTraceMiddleware",
    "message": "HttpRequest:\nPOST /\r\nHost: secretsmanager.us-east-1.amazonaws.com\r\nContent-Length: 2\r\nContent-Type: application/x-amz-json-1.1\r\nX-Amz-Target: secretsmanager.GetSecretValue\r\nUser-Agent: aws-sdk-kotlin/1.3.76 ua/2.1 api/secrets-manager#1.3.76 os/linux#5.10.227-239.884.amzn2.x86_64 lang/kotlin#2.0.21 md/javaVersion#21.0.5 md/jvmName#OpenJDK_64-Bit_Server_VM md/jvmVersion#21.0.5+11-LTS exec-env/AWS_Lambda_java21 m/E\r\nx-amz-user-agent: aws-sdk-kotlin/1.3.76\r\namz-sdk-invocation-id: 45c1d291-7636-4821-a045-cc147d4c597d\r\namz-sdk-request: attempt=1; max=3\r\nX-Amz-Date: 20241119T113825Z\r\nX-Amz-Security-Token: IQoJb3JpZ2luX2VjENz//////////wEaCXVzLWVhc3QtMSJHMEUCIQCiXueOAYOFl9/xE7X7Eka+W0k+k2Z3pCaVAJAsC+a9gQIgQ2r1WL/fyT1IplcEzUagfxEevz4cysinAGcyFJS01lIq9gIIdRACGgw0ODA0NjUzNDQwMjUiDK57AJPp23vu6ocLTSrTAgR9hip5wzMRkHenC4k+Atrfi4U2vSV5jG7DPJFVjBxObVj/EF8bUorO0Z8TnL5bDkzxYLhqyUY4Aqadxe6Rq1WX1RdAKERCbANgY4VKh/2diAKVG5gtgNExAZYdggHL43VVEF+KHStS7f4CsPMKathpMS6ptQqBJR+ZkqTKBSUymTW/vQGy0yPsB+piFtC6Cu80HzKeqLC+ISkLplHk/02lfYvmPta1cnuWAO9IGZOpY3eSqWdwQCIFK26Jt9e23fGdhhk5oT8AcHc8Rpzv8dGZ2R09dmiKJSR004iXuRLT6EoK4UV63k0nRIZCyvISM7c+XY3iDxACNbL2zwMqp1D/2Nj6zQdrSRs1Oh/TaOd3Nb+M/GQgSVaUlgkks4fo4tBnotNlaJUaE1XlLJadRIrZjrVb8VITllQXsLXBrBPnGNdwVxh6tE1MOG3VdmsfkBezjDCo8fG5BjqeAd25vtefOUaIvLcbWnlB4LNPbJ8+sGGCffMoYxn6XPj8slMIeKPKS+mwi8NbTZTYKxe/t0FuIeUVgnZXkpGpboBaHiwJ7nCqdwvoqM7BLOsq1bD4kzq2136EQf7qXuz2/4wEpu9K9qfpfvKNZTFudak+Iq+tsdirwdkUwwRwO2rjzK0u07pRvurA55u7bXFhsa+OKsERgt9+IEgGltGT\r\nAuthorization: AWS4-HMAC-SHA256 Credential=ASIAW7XPXKYM2SAQJAT7/20241119/us-east-1/secretsmanager/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;content-type;host;x-amz-date;x-amz-security-token;x-amz-target;x-amz-user-agent, Signature=30ae3038f0c3715489627b505cbb83d198b62deba261b3a27a909420b9ab8382\r\n\r\n{}",

    "level": "DEBUG",
    "loggerName": "httpTraceMiddleware",
    "message": "HttpResponse:\nHTTP 400: Bad Request\r\nconnection: close\r\ncontent-length: 152\r\ncontent-type: application/x-amz-json-1.1\r\ndate: Tue, 19 Nov 2024 11:38:27 GMT\r\nx-amzn-requestid: 66736b04-94c1-4563-bc1b-4f00a71a50f8\r\n\r\n{\"__type\":\"ValidationException\",\"message\":\"1 validation error detected: Value null at 'secretId' failed to satisfy constraint: Member must not be null\"}",

Steps to Reproduce

Using the following function definitions:

  override fun getSecretManager(): SecretsManagerClient {
    return SecretsManagerClient {
      logMode = LogMode.LogRequestWithBody + LogMode.LogResponseWithBody
      region = System.getenv("AWS_REGION")
      credentialsProvider = EnvironmentCredentialsProvider()
      endpointProvider = object : SecretsManagerEndpointProvider {
        override suspend fun resolveEndpoint(params: SecretsManagerEndpointParameters): Endpoint {
          return Endpoint(uri = "https://secretsmanager.$region.amazonaws.com")
        }
      }
    }
  }
  
private suspend fun getDatabaseSecret(secretArn: String): DatabaseSecret {
    log.info { "Secret: $secretArn" } // This logs the correct secret
    val secret = appComponent.getSecretManager().use { client ->
      val secretJson = client.getSecretValue {
        GetSecretValueRequest {
          secretId = secretArn
        }
      }
      log.info { "Secret: $secretJson" }
      Json.decodeFromString<DatabaseSecret>(secretJson.secretString!!)
    }
    return secret
  }

Possible Solution

Unknown

Context

The lambda runs in an isolated VPC and has the purpose connect to RDS and execute schema changes.
I setup the connection between the lambda and a vpc endpoint allowing connection to secret manager.

AWS SDK for Kotlin version

1.3.76

Platform (JVM/JS/Native)

JVM

Operating system and version

Lambda Runtime.JAVA_21

[lauzadis]

The empty JSON body {} at the end of your request log suggests that a null secretId is being configured on the request or not being sent some other way.

Can you confirm that the secretArn you're using is never null? I see in your code sample it's a non-null String, but would like to make sure

[niqo01]

Thank you for looking into this.
You can see in the code above that I added a log the secretArn and I can confirm it contains a non null value and the correct secret arn.

[lauzadis]

I'm unable to reproduce this issue. I can successfully invoke GetSecretValue and most importantly, I see that my request has a non-empty body:

HttpRequest:\nPOST /\r\nHost: secretsmanager.us-east-1.amazonaws.com\r\nContent-Length: 79\r\nContent-Type: application/x-amz-json-1.1\r\nX-Amz-Target: secretsmanager.GetSecretValue\r\nUser-Agent: aws-sdk-kotlin/1.3.67 ua/2.1 api/secrets-manager#1.3.67 os/macos#14.7.1 lang/kotlin#2.0.21 md/javaVersion#17.0.12 md/jvmName#OpenJDK_64-Bit_Server_VM md/jvmVersion#17.0.12+7-LTS m/E\r\nx-amz-user-agent: aws-sdk-kotlin/1.3.67\r\namz-sdk-invocation-id: dcced352-f010-4c1a-87c2-59acb0ea9099\r\namz-sdk-request: attempt=1; max=3\r\nX-Amz-Date: 20241119T203902Z\r\nX-Amz-Security-Token: IQoJb3JpZ2luX2VjEOX//////////wEaCXVzLWVhc3QtMSJHMEUCIQDHd1nwjn0wsITuAx/nAgpe4nag0HvGnf10NlpOd/YVoAIgYVLnTPpyx80MrSKBcAyI1dL5V65uQNx8KxtR+CpdCkgqngIIfhAAGgwxNjQ3MTEyODA5NzciDPWrgYfYyjuWTrs6tir7AayaU3OWY15P8+6Hml/hRmJobKJDnTRQPDAtOnQM36q5dmqDw5WMrUt+nN272HxYvwcVKe0Rp60YF4WViUsSGpvmFFKQRSQdd2kbaIqU8hF3fr/ghEz1Zd62b3YglKiAWXEMXofTRNi4iJU/AoWzQ9t8VXhhAD3mP1aAB30FLrpk7rliWylI8Xq+PX0QB+L9VGjs4EKf8IDAwXJukN9nzqKHMoyUCz1r1VOClpH+yQuH3zXOUMqmLPYM0GkiDsyH8dzi6AEMbpkoSnohWl2CYfUcG+TO1k/1JNE/2O69YtU4Rmj2inrEa2u/hZsj33nvNOlbhETxMDDH4VYhMJns87kGOp0Bc3usMxB0EXtVLbWYIQQmMRncF12B7JNoyOju6XvRhrN3vYCozYrKhGf3IV0sgZSbqPrma/FPAP/SlyozSRwj7HP4ScVUhiO+DEOqE2/ICpqhGoOFMKcQVV+TMDJgJE3DYbfskS29ghAkzcAmTxy4GDrdf0uUOw2ei+qh49s43ptmBgR4Zt8b1TbuqIGdc5Heeqep2YRIrKVlQf3PeA==\r\nAuthorization: AWS4-HMAC-SHA256 Credential=ASIASMWMOWFI6RTXK5W7/20241119/us-east-1/secretsmanager/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;content-type;host;x-amz-date;x-amz-security-token;x-amz-target;x-amz-user-agent, Signature=79e78761625cbb7c5a05c5bcf06dab36f2c378dd9b8c7feca6575b6c818e649f\r\n\r\n{"SecretId":"arn:aws:secretsmanager:us-east-1:<ACCOUNT_ID_REDACTED>:secret:test-QaCrio"}

The fact that your request has an empty body {} strongly suggests you did send a request with a null secretId... I can't think of another reason this would fail for you.

Are you able to consistently reproduce this issue, can you provide a minimal reproduction?

[niqo01]

I can reproduce consistently and was able to create a minimal reproduction project:
secret-issue.zip

Once uncompress, run ./gradlew :infra:libs:db-migration:test
(Note that you might need valid default aws credentials, or set your own configuration in infra/libs/db-migration/src/test/kotlin/TestComponent::getSecretManager)

This should output a link to the test result with the issue or you can visit infra/libs/db-migration/build/test-results folder.
In this test result you should see the same error message.:
1 validation error detected: Value null at 'secretId' failed to satisfy constraint: Member must not be null"

[lauzadis]

Hey @niqo01, thanks for the reproduction, I've figured out the issue. You are creating a GetSecretValueRequest inside the DSL builder which is not used:

      // GetSecretValueRequest is created inside the DSL builder, but not used by the SDK...
      val secretJson = client.getSecretValue {
        GetSecretValueRequest {
          secretId = secretArn
        }
      }

You should update your request to either:

      // 1. Configure the DSL builder
      val secretJson = client.getSecretValue {
          secretId = secretArn
      }

      // 2. Pass a request (note: parentheses instead of braces)
      val secretJson = client.getSecretValue(GetSecretValueRequest {
          secretId = secretArn
      })

Sorry for not catching this in the first code sample you posted, it's hard to spot! Let me know if there's any other issues you come across here, otherwise we'll be closing the issue.

[niqo01]

Oh, I am sorry I didn't catch that either. Thank you for looking into it and figuring this out. Greatly appreciated.

If I may, why not having the builder function from GetSecretValueRequest check if a secretId has been set and raise an issue right there?

[lauzadis]

That's a great question and one we've thought about a lot in the early days of the SDK, see this discussion post for more context: #365

For a short summary, services model required fields using Smithy's @required trait (and Secrets Manager correctly marks secretId as @required). However, removing @required from a field is a valid model mutation and not considered a breaking change. But, it would break our generated Kotlin clients since the type would change from String to String?. For that reason we've decided to effectively ignore @required for operation inputs and delegate validation to the service.