(cloudformation-include): cdk synth fails on CfnInclude when using ref on SAM generated resource

[jluttrell]

General Issue

cdk synth fails on CfnInclude when using ref on sam generated resource

The Question

We use cdk synth to compile separate smaller templates into a single deployable stack template and then deploy using sam deploy. Some templates reference the SAM autogenerated AWS::Lambda::Version via the referenceable property <function‑LogicalId>.Version as defined in the sam docs. When we use cdk.CfnInclude to include the template, the cdk errors with Error: Element used in Ref expression with logical ID: '<function‑LogicalId>.Version' not found.

Example (simplified) template:

AWSTemplateFormatVersion: "2010-09-09"
Transform: "AWS::Serverless-2016-10-31"

Resources:
  MyServerlessFunction:
    Type: "AWS::Serverless::Function"
    Properties:
      AutoPublishAlias: myalias
      FunctionName: MyFunction
      CodeUri: index.js
  GGCoreLambdaComponent:
    Type: AWS::GreengrassV2::ComponentVersion
    Properties:
      LambdaFunction:
        LambdaArn: !Ref MyServerlessFunction.Version #this is the failing line

This would generate the error Error: Element used in Ref expression with logical ID: 'MyServerlessFunction.Version' not found

I understand why it's erroring and that the cdk doesn't fully support SAM and/or vice versa, but I'm wondering if there is any other work around. I've tried using --ignore-errors and --validation false when doing cdk synth but the error remains.

If helpful, the cdk code is the following:

import * as cdk from "aws-cdk-lib";
import { CfnInclude } from "aws-cdk-lib/cloudformation-include";
import { Construct } from "constructs";
import { readdirSync, existsSync } from "fs";

const stackName = "GreengrassComponentsNode";

const entries = readdirSync("./components", { withFileTypes: true })
  .filter(
    (dirent) =>
      dirent.isDirectory() &&
      existsSync(`./components/${dirent.name}/template.yml`)
  )
  .map((dirent) => {
    return {
      name: dirent.name,
      template: `./components/${dirent.name}/template.yml`,
    };
  });

class GreengrassComponentsStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    new CfnInclude(this, "BaseTemplate", {
      templateFile: "./template.yml",
    });

    entries.forEach((entry) => {
      new CfnInclude(this, entry.name, {
        templateFile: entry.template,
      });
    });
  }
}

const app = new cdk.App();

new GreengrassComponentsStack(app, stackName);

CDK CLI Version

2.18.0

Framework Version

No response

Node.js Version

16.6.2

OS

macOS, linux

Language

Typescript

Language Version

TypeScript (4.5.4)

Other information

Stacktrace

Error: Element used in Ref expression with logical ID: 'MyServerlessFunction.Version' not found
    at CfnParser.parseIfCfnIntrinsic (<redacted>/node_modules/aws-cdk-lib/core/lib/cfn-parse.ts:525:19)
    at CfnParser.parseValue (<redacted>/node_modules/aws-cdk-lib/core/lib/cfn-parse.ts:494:33)
    at CfnParser.parseValue (<redacted>/node_modules/aws-cdk-lib/core/lib/cfn-parse.ts:500:25)
    at CfnParser.parseValue (<redacted>/node_modules/aws-cdk-lib/core/lib/cfn-parse.ts:500:25)
    at Function._fromCloudFormation (<redacted>/node_modules/aws-cdk-lib/aws-greengrassv2/lib/greengrassv2.generated.ts:153:51)
    at CfnInclude.getOrCreateResource (<redacted>/node_modules/aws-cdk-lib/cloudformation-include/lib/cfn-include.ts:649:40)
    at new CfnInclude (<redacted>/node_modules/aws-cdk-lib/cloudformation-include/lib/cfn-include.ts:142:12)
    at <redacted>/gg-components-stack.ts:30:7
    at Array.forEach (<anonymous>)
    at new GreengrassComponentsStack (<redacted>/gg-components-stack.ts:29:13)

Other possible workarounds

These are pretty specific to our use case but:

• Define the AWS::Lambda::Version ourselves instead of having SAM autogenerate it and then reference to that resource should work
• Define the lambda arn with version ourself (i.e. LambdaArn: !Join [":", [!GetAtt MyServerlessFunction.Arn, "1"]])

But we'd like to avoid these more error prone and manual steps.

[skinny85]

Hey @jluttrell,

thanks for opening the issue. This is indeed a very tricky situation.

I think you should be able to unblock yourself by downloading the processed template from the CloudFormation console, and including that. Now, that will make cdk diff show a million changes. However! I believe running cdk deploy should actually not replace anything, and just be a no-op. But, to make sure I'm right, please test it on a development environment before doing this in production!

But there's actually a problem, because a no-op deployment will not update the template, so cdk diff ran after the deployment will still show the million changes. So, you need to make any change to the Stack so that it gets updated. The simplest thing to do is to change the Metadata of any resource:

    cfnInclude = new CfnInclude(this, "BaseTemplate", {
      templateFile: "./template.yml",
    });
    const r = cfnInclude.getResource('MyServerlessFunction');
    r.addMetadata('A', 'B');

So, to sum up:

1. Download the processed template from the AWS CloudFormation Console.
2. Include the processed template with CfnInclude.
3. Run cdk diff, see a million changes.
4. Make any change (like the metadata) to the Stack.
5. Run cdk deploy MyTestStack (in a test environment, not production!). It should not replace any resources, and should complete successfully.
6. Run cdk diff again. It should now show no changes.
7. Remove the Metadata added in step 4.

Hope this helps!

Thanks,
Adam

[jluttrell]

hi @skinny85, thanks for the quick response!

I understand the process you've outlined. Indeed, pre-processing the individual templates with SAM would solve the issue. However, this process runs in our CI/CD pipeline so manual step of downloading processed template isn't possible. As far as I remember, SAM requires the template to be processed in the cloud. So I'm not sure how I could pre-process all the child templates and then use cdk to include the already processed templates. But I'll look into if/how that's possible.

To give brief background on what we're trying to achieve, we want individuals to create greengrass core components (and required aws resources such as iam roles/policies, dynamo tables, sqs queues, etc) in this mono repo using cloudformation templates they're familiar with and then compiling them all into a single template (automated using the cdk) for deployment to a single stack.

We could use only SAM (and no cdk) and have a "master" template that directly references each of the individual component templates as AWS::Serverless::Application.

i.e.

  ComponentOne:
    Type: AWS::Serverless::Application
    Properties:
      Location: ./components/ComponentOne/template.yml

  ComponentTwo:
    Type: AWS::Serverless::Application
    Properties:
      Location: ./components/ComponentTwo/template.yml

And this wouldn't be a terrible solution but this would require a manual update of the child template path in the master template for each created/deleted template and I thought we'd try and be clever using the cdk to automate that 😎 Which makes me think we may be able to use the cdk to add the child templates as AWS::Serverless::Applications and then deploy that 🤔 But at that point we may not even need the cdk to compile that master template. I'll let you know what we find out.

More in regards to the cdk, are there any plans to better integrate with SAM, perhaps not in processing the template but at least parsing SAM schema (such as any of its referenceable properties on autogenerated resources) as valid? So that we could use <function‑LogicalId>.Version mentioned in the issue without cdk synth failing? I understand it's a complicated problem because those would required sam processing... thanks again!

[jluttrell]

Got it to work by including the child templates as AWS::Serverless::Application. The cdk code is now the following:

import * as cdk from "aws-cdk-lib";
import { CfnInclude } from "aws-cdk-lib/cloudformation-include";
import { aws_sam as sam } from "aws-cdk-lib";
import { Construct } from "constructs";
import { readdirSync, existsSync } from "fs";

const stackName = "GreengrassComponentsNode";

const entries = readdirSync("./components", { withFileTypes: true })
  .filter(
    (dirent) =>
      dirent.isDirectory() &&
      existsSync(`./components/${dirent.name}/template.yml`)
  )
  .map((dirent) => {
    return {
      name: dirent.name,
      template: `./components/${dirent.name}/template.yml`,
    };
  });

class GreengrassComponentsStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    new CfnInclude(this, "BaseTemplate", {
      templateFile: "./template.yml",
    });

    entries.forEach((entry) => {
      new sam.CfnApplication(this, entry.name, {
        location: entry.template,
      });
    });
  }
}

const app = new cdk.App();

new GreengrassComponentsStack(app, stackName);

Now the child templates are created as nested stacks but that may be even better than all as a single stack. Thanks for the help!

[skinny85]

I'm glad you got it working @jluttrell!

As far as I remember, SAM requires the template to be processed in the cloud. So I'm not sure how I could pre-process all the child templates and then use cdk to include the already processed templates. But I'll look into if/how that's possible.

I meant downloading the processed template from the CloudFormation console:

Not sure if there's a way to do it through the CloudFormation API though.

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.