Using CDK variables in templates

[andyRokit]

Does anybody have any tricks for using strings from CDK constructs in templates?

Example
I'd like to create some API Gateways using a SpecRestApi.

I have some resources that I'd like to include in some of these Gateways, like the ID of a VpcLink or a BaseUrl for a VPC Endpoint (created in another construct).

What I've got so far

I've created a helper class to:

• Create an API definition from an uploaded asset
• Hardcode LogicalIds for constructs

import { CfnResource, Fn } from 'aws-cdk-lib'
import { Construct } from 'constructs'
import { Asset } from 'aws-cdk-lib/aws-s3-assets'

import * as apigateway from 'aws-cdk-lib/aws-apigateway'


export default class CdkHelper {
    /**
     * CDK generates logical IDs for CF resources, which means we can't depend on them in templates (such as OpenApi sepcs)
     * This method forces CDK to give these a fixed ID in the templates.
     */
    static hardcodeLogicalId(construct: Construct, newLogicalId: string) {
        const resource = construct.node.defaultChild as CfnResource
        resource.overrideLogicalId(newLogicalId)
    }

    /**  
     * Creates an ApiDefinition from an OpenApi spec file.
     * Enables use of CloudFormation variables (such as ARNs) in the file.
     */
     static createApiDefinition(scope: Construct, specPath: string) : apigateway.ApiDefinition {
        // Upload spec to S3
        const originalSpec = new Asset(scope, 'originalAsset', {
            path: specPath
        })

        // Pulls the content back into the template.  Being inline, this will now respect CF references within the file.
        const transformedSpec = Fn.transform('AWS::Include', {
            'Location': originalSpec.s3ObjectUrl
        })

        return apigateway.AssetApiDefinition.fromInline(transformedSpec)
    }
}

So you can do things like:

const fooRole = new iam.Role(this, `${props.prefix}-Role`, {
    assumedBy: new iam.ServicePrincipal('apigateway.amazonaws.com'),
    inlinePolicies: {
           ...
        })
    }
})

CdkHelper.hardcodeLogicalId(fooRole, 'fooRole')

const apiDefinition = CdkHelper.createApiDefinition(this, 'assets/foo/openapi-spec.yaml')

...then in the spec

x-amazon-apigateway-integration:
  credentials:
    Fn::Sub: "${fooRole.Arn}"

But it's not as easy to do with non-resource strings or even string via parameters. E.g.

import { Construct } from 'constructs'
import CdkHelper from '../helpers/cdk-helper'

import * as apigateway from 'aws-cdk-lib/aws-apigateway'


export interface CustomerGatewayProps {
    readonly prefix: string,
    readonly privateGatewayBaseUrl: string,
    readonly vpcLink: apigateway.VpcLink
}

export class CustomerGateway extends Construct {
    public api: apigateway.IRestApi

    constructor(scope: Construct, id: string, props: CustomerGatewayProps) {
        super(scope, id)


        const apiDefinition = CdkHelper.createApiDefinition(this, 'assets/gateways/customer/openapi-spec.yaml')

        // How do I get vpcLink ID and privateGatewayBaseUrl into CF variables so I can use them in the spec?

        this.api = new apigateway.SpecRestApi(this, `${props.prefix}-Customer-Gateway`, {
            apiDefinition: apiDefinition
        })
    }
}

I've adapted my current CDK-workarounds from items I've found from other CDK users. I'm hoping that there are a few more created ideas out there to get me over this next hurdle!

Edit
These workarounds are really in place to overcome a CloudFormation issue: they don't allow DefinitionSubstitutions in all their resources which take a body. I've raised an issue on their Public Roadmap - feel free to head over there and give it an upvote!

[andyRokit]

Typical. Spent days trying to work round the issue only to come up with the solution a few hours after posting this. Oh well. Here we go:

Use NestedStack stacks with CfnParameter (rather than let CDK try to handle the variables internally). That gives you a CloudFormation reference for them.

Example (continued from above)

import { Construct } from 'constructs'
import CdkHelper from '../helpers/cdk-helper'

import * as apigateway from 'aws-cdk-lib/aws-apigateway'
import { CfnParameter, NestedStack, NestedStackProps } from 'aws-cdk-lib'


export interface CustomerGatewayProps extends NestedStackProps {
    readonly prefix: string
}

export class CustomerGateway extends NestedStack {
    public api: apigateway.IRestApi

    constructor(scope: Construct, id: string, props: CustomerGatewayProps) {
        super(scope, id, props)

        new CfnParameter(this, 'vpcLinkId', {type: 'String'})
        new CfnParameter(this, 'privateGatewayBaseUrl', {type: 'String'})
        new CfnParameter(this, 'orderServiceApiId', {type: 'String'})


        const apiDefinition = CdkHelper.createApiDefinition(this, 'assets/gateways/customer/openapi-spec.yaml')

        this.api = new apigateway.SpecRestApi(this, `${props.prefix}-Customer-Gateway`, {
            apiDefinition: apiDefinition
        })
    }
}

...pass the params in using the Cfn (rather than CDK) mechanism:

const customerGateway = new CustomerGateway(this, 'customerGateway', {
      prefix: this.stackName,
      parameters: {
        vpcLinkId: network.vpcLink.vpcLinkId,
        privateGatewayBaseUrl: network.privateGatewayBaseUrl,
        orderServiceApiId: orderService.api.restApiId
      }
    })

..and that allows you to use them in your templates:

x-amazon-apigateway-integration:
 type: http_proxy
 connectionType: VPC_LINK
 connectionId:
   Ref: vpcLinkId
 uri:
   Fn::Sub: ${privateGatewayBaseUrl}/prod/orders
 requestParameters:
   integration.request.header.x-apigw-api-id:
     Fn::Sub: "'${orderServiceApiId}'"
 httpMethod: POST
 passthroughBehavior: when_no_match
 responses:
   default:
     statusCode: '201'

[andyRokit]

With this approach you no longer have to hard code logical ids. Once you make the child a NestedStack it makes CDK use parameters to pass string values (rather than internal references). If you're having trouble with this:

• Add the parameters to the stack first, without using them anywhere
• Run cdk synth
• Look in your CDK folder for the template which corresponds to your NestedStack. Should be something like: MyProjectMyNestedStackA1B2C3D4.nested.template.json

You should see your parameters. You can now use those references exactly as if you had added them via CloudFormation.

[davidsteed1]

Thanks for the prompt reply. I did that and in the nested stack I am getting this in the nested stack

{
 "Parameters": {
  "userpoolid": {
   "Type": "String"
  },

and am putting

x-amazon-apigateway-authorizer:
        providerARNs:
          - Fn::Sub: arn:aws:cognito-idp:${AWS::Region}:${AWS::AccountId}:userpool/{userpoolid}
        type: cognito_user_pools

And it is not working. When I change {userpoolid} to the actual value it works

The error just says it was unable to deploy api gateway and does not say why, but it must be this

[davidsteed1]

I am missing the $

[davidsteed1]

Thanks. I have to say this is a genius solution. Not sure I would have managed to figure this out.

[u7w4qv5PiZ5NOXMM69r5vPeLl4GZ4KzycIk1eKX]

I'm glad, that i'm not the only one using this approach...

I find it a bit frustrating how CDK is abstracting CloudFormation. The pattern of using native CloudFormation parameters is deemed an anti-pattern in CDK: "In general, we recommend against using AWS CloudFormation parameters with the AWS CDK. The usual ways to pass values into AWS CDK apps are context values and environment variables...". I understand why, but it makes me question the entire logic design of CDK.

Edge-Case: You have a nested stack A, where you construct a (S3) Bucket. You define the BucketName as an output. In your nested stack B you need to reference said Bucket from nested stack A for constructing a BucketDeployment, therefore using the aforementioned output of stack A as a parameter for stack B. This will not be possible, since BucketDeployment requires a reference to an IBucket (L2) object, which can't be constructed from a (L1) CfnParameter.valueAsString.

A solution could be pretty straightforward, if CDK adhered more to the limitations of vanilla CloudFormation:

1. have a generic bootstrap lambda function for transferring assets from the CDK bucket to a destination bucket
2. Make any bucket deployment asset part of the CDK assets upload
3. automagically add a custom resource to a constructed stack whenever a bucket deployment is constructed, which triggers the bootstrap lambda function and transfers the asset.

EDIT: Just figured out i can implement this pattern myself [2] 🥳

instead CDK does this naughty local aws s3 sync calls... Side-Effects... 🤮

[1] https://docs.aws.amazon.com/cdk/v2/guide/parameters.html
[2] https://docs.aws.amazon.com/cdk/v2/guide/assets.html

[andyRokit]

Looks like this issue has just been closed by the inactive bot!

For anyone who had read this far and was hoping for something better that my suggested workaround of using a nested stack, please upvote this CloudFormation issue to address the underlying cause.

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.