Principle of most surprise: Secret.fromSecretNameV2, and a V3 proposal.

[ClaudiuCeia]

Hey awesome CDK team,

I'll preface this by saying I love the work you're all doing, and I understand it's a gargantuan task getting CDK to play nice on top of AWS APIs and what not. I've run into deadly embraces and other non-trivial issues before, and I can generally understand how and why they occur, and why it's difficult to address them. But this newest issue really took me by surprise.

---

Starting this as a discussion at the end of hours of trying to figure out what kind of permissions I'm lacking using Secret.fromSecretNameV2. For most of you who have used it, you probably can already tell what I'm talking about.

The problem

My use case, to isolate it to a minimum, involves a third party service which provides two pieces of information, a client ID, and a secret key. I'm storing these as secrets, namely: ${environment}/foo-client-id and ${environment}/foo-client-secret. I then feed these into a Fargate service:

  const service = new ApplicationLoadBalancedFargateService(
      this,
      `${environment}-service`,
      {
       // ...
        taskImageOptions: {
          // ...
          environment: {
            // ...
          },
          secrets: {
            THIRD_PARTY_CLIENT_ID: EcsSecret.fromSecretsManager(Secret.fromSecretNameV2(
              this,
              `${environment}-foo-client-id`,
              `${environment}/foo-client-id`,
            )),
            THIRD_PARTY_CLIENT_SECRET: EcsSecret.fromSecretsManager(Secret.fromSecretNameV2(
              this,
              `${environment}-foo-client-secret`,
              `${environment}/foo-client-secret`,
            )),
          },
        },
      },
    );

At this point, I already had a different stack which only needed THIRD_PARTY_CLIENT_ID. Deployment for that went without a hitch and has been running for a while. Imagine my surprise when this new stack fails due to authorization issues regarding THIRD_PARTY_CLIENT_SECRET. All I could find were suggestions to use the full ARN (which has a generated suffix) and generally very ugly solutions for something that already works in a different stack.

The surprise

Eventually I got around to digging into the source code. I never could have imagined to find this gem:

If your secret name ends with a hyphen and 6 characters, you should always use fromSecretCompleteArn() to avoid potential AccessDeniedException.

Sorry, but what in the name of black magic is this? Looking at the documentation provides further insight, and I understand CDK is built on top of existing tech and has to maintain compatibility, but surely there has to be a way to better deal with this?

I also understand that the name is a component of the ARN, but in this case implementation details (the suffix appended by Secret Manager) gets in the way - I expect that the name I pass to this function is the name I also see in Secret Manager.

My solution

As revenge for my wasted hours, I just went ahead and renamed ${environment}/foo-client-secret to ${environment}/foo-client-secretv2, thus avoiding it be mistaken for a suffix. Sorry, I'm petty. I understand there are probably many cases where this is not something that can be done, but it's very low effort for cases where it can be - and would be worth mentioning at least.

A better solution

Static typing

CDK is Typescript native, for many cases the type below would guard against this issue, because the real type of the name is not string, but string except any string with a "hyphen-6" suffix:

export type Split<
  T extends number | string,
  R extends string[] = []
> = `${T}` extends `${infer head}${infer tail}` ? Split<tail, [...R, head]> : R;

type Hyphen6<
  T extends string,
  Cache extends string = T,
> = T extends `${infer _}-${infer Suffix}`
  ? Split<Suffix>["length"] extends 6
    ? "Secret names cannot end with a hyphen and 6 characters"
    : Hyphen6<Suffix, Cache>
  : Cache;

const fromSecretNameV3 = <T extends string>(secretName: Hyphen6<T>): 42 => {
  return 42;
};

fromSecretNameV3("abc");  // ok
fromSecretNameV3("abc-123"); // ok
fromSecretNameV3("abc-123456");  // error
fromSecretNameV3(`${environment}-abc-123456`); // error

Of course it doesn't catch every case, but I think often enough people would pass a literal type here.

Runtime

Considering how easy it is to run into this, a runtime warning for secret names that end with a matching "hyphen-6" suffix could easily be provided. Furthermore - I'm not really sure that I understand when it would make sense to allow these to go through in the first place. But I don't have the energy to dive into it at the moment. I'm sure a warning in the console would do more good than harm though.

Thank you for listening to my Ted talk.