CloudFormation stack creation failure

[ejjordan]

CodeCommit is no longer available to new users, leading to the following error during stack formation:
CreateRepository request is not allowed because there is no existing repository in this AWS account or AWS Organization (Service: AWSCodeCommit; Status Code: 400; Error Code: OperationNotAllowedException;

I am happy to provide more information if needed.

Related to #33

[ejjordan]

Below is the generated template for protein-folding-container-xxxx.
The issue seems to be that CodeBuildProjectDownload still has a dependency on CodeCommit, via CodeRepository.

AWSTemplateFormatVersion: 2010-09-09
Description: 'batch-protein-folding-cfn-container.yaml: Creates container building
  and hosting resources.'
Parameters:
  ApplicationName:
    Description: Name of the application, if applicable
    Type: String
    Default: Unknown
  Environment:
    Description: Leave this as "main" unless you are testing a pre-release versions
      of this architecture.
    Type: String
    Default: main
    AllowedValues:
    - main
    - dev
  DownloadFsxData:
    Description: Automatically populate FSx for Lustre file system with model parameters
      and sequence databases?
    Type: String
    Default: Y
    AllowedValues:
    - Y
    - N
  CodeRepoS3BucketName:
    Description: S3 Bucket Name to retrieve code zip
    Type: String
    Default: aws-hcls-ml
  S3BucketARN:
    Description: S3 Bucket ARN
    Type: String
  Timestamp:
    Description: Timestamp for the cfn deployment
    Type: Number
    Default: 9999999999
Conditions:
  DownloadFsxDataCondition:
    Fn::Equals:
    - Ref: DownloadFsxData
    - Y
Resources:
  CodeRepository:
    Type: AWS::CodeCommit::Repository
    Properties:
      Code:
        BranchName: main
        S3:
          Bucket:
            Ref: CodeRepoS3BucketName
          Key:
            Fn::Sub: ${Environment}/batch-protein-folding.zip
      RepositoryDescription: Code for running batch protein folding on AWS
      RepositoryName:
        Fn::Join:
        - '-'
        - - batch-protein-folding
          - code-repo
          - Fn::Select:
            - 4
            - Fn::Split:
              - '-'
              - Fn::Select:
                - 2
                - Fn::Split:
                  - /
                  - Ref: AWS::StackId
      Tags:
      - Key: Application
        Value:
          Ref: ApplicationName
      - Key: StackId
        Value:
          Ref: AWS::StackId
      - Key: SolutionId
        Value: SO9164
  DownloadContainerRegistry:
    Type: AWS::ECR::Repository
    Properties:
      EncryptionConfiguration:
        EncryptionType: AES256
      ImageScanningConfiguration:
        ScanOnPush: true
      Tags:
      - Key: Application
        Value:
          Ref: ApplicationName
      - Key: StackId
        Value:
          Ref: AWS::StackId
      - Key: SolutionId
        Value: SO9164
  CodeBuildRole:
    Type: AWS::IAM::Role
    Properties:
      Description: Required service policies to support building containers
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - codebuild.amazonaws.com
          Action:
          - sts:AssumeRole
      ManagedPolicyArns:
      - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryPowerUser
      Policies:
      - PolicyName: CodeBuildPolicy
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
          - Effect: Allow
            Action:
            - logs:CreateLogGroup
            - logs:CreateLogStream
            - logs:PutLogEvents
            Resource:
            - Fn::Sub: arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/codebuild/CodeBuildProject*
          - Effect: Allow
            Action:
            - s3:PutObject
            - s3:GetObject
            - s3:GetObjectVersion
            - s3:GetBucketAcl
            - s3:GetBucketLocation
            Resource:
            - Fn::Join:
              - '-'
              - - arn:aws:s3:::codepipeline
                - Ref: AWS::Region
                - '*'
            - Fn::Sub: ${S3BucketARN}*
            - Fn::Sub: arn:aws:s3:::${CodeRepoS3BucketName}/*
          - Effect: Allow
            Action:
            - s3:GetObject
            - s3:GetObjectVersion
            - s3:GetBucketAcl
            - s3:GetBucketLocation
            Resource:
            - Fn::Sub: arn:aws:s3:::${CodeRepoS3BucketName}*
          - Effect: Allow
            Action:
            - codecommit:GitPull
            Resource:
            - Fn::Join:
              - ':'
              - - arn:aws:codecommit
                - Ref: AWS::Region
                - Ref: AWS::AccountId
                - Fn::GetAtt:
                  - CodeRepository
                  - Name
          - Effect: Allow
            Action:
            - codebuild:CreateReportGroup
            - codebuild:CreateReport
            - codebuild:UpdateReport
            - codebuild:BatchPutTestCases
            - codebuild:BatchPutCodeCoverages
            Resource:
            - Fn::Sub: arn:aws:s3:::codebuild:${AWS::Region}:${AWS::AccountId}:report-group/batch-protein-folding*
      Tags:
      - Key: Application
        Value:
          Ref: ApplicationName
      - Key: StackId
        Value:
          Ref: AWS::StackId
      - Key: SolutionId
        Value: SO9164
  EncryptionKey:
    Type: AWS::KMS::Key
    Properties:
      EnableKeyRotation: true
      KeyPolicy:
        Version: 2012-10-17
        Id: batch-protein-folding-key
        Statement:
        - Sid: Enable IAM User Permissions
          Effect: Allow
          Principal:
            AWS:
              Fn::Join:
              - ':'
              - - 'arn:aws:iam:'
                - Ref: AWS::AccountId
                - root
          Action:
          - kms:Create*
          - kms:Describe*
          - kms:Enable*
          - kms:List*
          - kms:Put*
          - kms:Update*
          - kms:Revoke*
          - kms:Disable*
          - kms:Get*
          - kms:Delete*
          - kms:TagResource
          - kms:UntagResource
          - kms:ScheduleKeyDeletion
          - kms:CancelKeyDeletion
          Resource: '*'
        - Sid: Enable CodeBuild Encryption
          Effect: Allow
          Principal:
            AWS:
              Fn::GetAtt:
              - CodeBuildRole
              - Arn
          Action:
          - kms:Encrypt
          - kms:Decrypt
          - kms:ReEncrypt*
          - kms:GenerateDataKey*
          - kms:DescribeKey
          Resource: '*'
        - Sid: Enable CloudWatch Logs Encryption
          Effect: Allow
          Principal:
            Service: logs.amazonaws.com
          Action:
          - kms:Encrypt
          - kms:Decrypt
          - kms:ReEncrypt*
          - kms:GenerateDataKey*
          - kms:DescribeKey
          Resource: '*'
      Tags:
      - Key: Application
        Value:
          Ref: ApplicationName
      - Key: StackId
        Value:
          Ref: AWS::StackId
      - Key: SolutionId
        Value: SO9164
  CodeBuildLambdaExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - lambda.amazonaws.com
          Action:
          - sts:AssumeRole
      Policies:
      - PolicyName: StartCodeBuildLambdaPolicy
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
          - Effect: Allow
            Action:
            - codebuild:StartBuild
            Resource: '*'
      ManagedPolicyArns:
      - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
  StartCodeBuildLambda:
    Type: AWS::Lambda::Function
    Properties:
      ReservedConcurrentExecutions: 10
      Code:
        ZipFile: "import logging          \nimport cfnresponse\nimport boto3\nLOGGER\
          \ = logging.getLogger()\nLOGGER.setLevel(logging.INFO)\ndef lambda_handler(event,\
          \ context):\n    try:\n        LOGGER.info('REQUEST RECEIVED:\\n %s', event)\n\
          \        LOGGER.info('REQUEST RECEIVED:\\n %s', context)\n        if event['RequestType']\
          \ == 'Create':\n            LOGGER.info('CREATE!')\n            client =\
          \ boto3.client('codebuild')\n            project_name = event['ResourceProperties']['ProjectName']\n\
          \            response = client.start_build(projectName=project_name)\n \
          \           cfnresponse.send(event, context, cfnresponse.SUCCESS, {\"response\"\
          :\"Resource creation successful!\"})\n        elif event['RequestType']\
          \ == 'Update':\n            LOGGER.info('UPDATE!')\n            client =\
          \ boto3.client('codebuild')\n            project_name = event['ResourceProperties']['ProjectName']\n\
          \            response = client.start_build(projectName=project_name)   \
          \                   \n            cfnresponse.send(event, context, cfnresponse.SUCCESS,\
          \ {\"response\":\"Resource update successful!\"})\n        elif event['RequestType']\
          \ == 'Delete':\n            LOGGER.info('DELETE!')\n            cfnresponse.send(event,\
          \ context, cfnresponse.SUCCESS, {\"response\":\"Resource deletion successful!\"\
          })\n        else:\n            LOGGER.info('FAILED!')\n            cfnresponse.send(event,\
          \ context, cfnresponse.FAILED, {\"response\":\"Unexpected event received\
          \ from CloudFormation\"})\n    except: \n        LOGGER.info('FAILED!')\n\
          \        cfnresponse.send(event, context, cfnresponse.FAILED, {\"response\"\
          :\"Exception during processing\"})\n"
      Description: Start CodeBuildProject
      Handler: index.lambda_handler
      MemorySize: 512
      Role:
        Fn::GetAtt: CodeBuildLambdaExecutionRole.Arn
      Runtime: python3.10
      Timeout: 10
      Tags:
      - Key: Application
        Value:
          Ref: ApplicationName
      - Key: StackId
        Value:
          Ref: AWS::StackId
      - Key: SolutionId
        Value: SO9164
  DeleteContainerRepoLambdaExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - lambda.amazonaws.com
          Action:
          - sts:AssumeRole
      Policies:
      - PolicyName: DeleteContainerRepoLambdaPolicy
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
          - Effect: Allow
            Action:
            - ecr:DeleteRepository
            Resource: '*'
      ManagedPolicyArns:
      - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
      - arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole
  DeleteContainerRepoLambda:
    Type: AWS::Lambda::Function
    Properties:
      ReservedConcurrentExecutions: 10
      Code:
        ZipFile: "import logging          \nimport cfnresponse\nimport boto3\n\nLOGGER\
          \ = logging.getLogger()\nLOGGER.setLevel(logging.INFO)\n\ndef lambda_handler(event,\
          \ context):\n    try:\n        LOGGER.info('REQUEST RECEIVED:\\n %s', event)\n\
          \        LOGGER.info('REQUEST RECEIVED:\\n %s', context)\n        if event['RequestType']\
          \ == 'Create':\n            LOGGER.info('CREATE!')\n            cfnresponse.send(event,\
          \ context, cfnresponse.SUCCESS, {\"response\":\"Resource creation successful!\"\
          })\n        elif event['RequestType'] == 'Update':\n            LOGGER.info('UPDATE!')\n\
          \            cfnresponse.send(event, context, cfnresponse.SUCCESS, {\"response\"\
          :\"Resource update successful!\"})\n        elif event['RequestType'] ==\
          \ 'Delete':\n            LOGGER.info('DELETE!')\n            ecr = boto3.client('ecr')\n\
          \            repo_name = event['ResourceProperties']['ContainerRepo']\n\
          \            repo_deletion_response = ecr.delete_repository(\n         \
          \     repositoryName=repo_name,\n              force=True\n            \
          \  )\n            LOGGER.info(f\"Repo deletion response:\\n{repo_deletion_response}\"\
          )\n            cfnresponse.send(event, context, cfnresponse.SUCCESS, {\"\
          response\":\"Resource deletion successful!\"})\n        else:\n        \
          \    LOGGER.info('FAILED!')\n            cfnresponse.send(event, context,\
          \ cfnresponse.FAILED, {\"response\":\"Unexpected event received from CloudFormation\"\
          })\n    except: \n        LOGGER.info('FAILED!')\n        cfnresponse.send(event,\
          \ context, cfnresponse.FAILED, {\"response\":\"Exception during processing\"\
          })\n"
      Description: Delete ECR repository
      Handler: index.lambda_handler
      MemorySize: 512
      Role:
        Fn::GetAtt: DeleteContainerRepoLambdaExecutionRole.Arn
      Runtime: python3.10
      Timeout: 10
      Tags:
      - Key: Application
        Value:
          Ref: ApplicationName
      - Key: StackId
        Value:
          Ref: AWS::StackId
      - Key: SolutionId
        Value: SO9164
  CodeBuildProjectDownload:
    Type: AWS::CodeBuild::Project
    Properties:
      Artifacts:
        Type: NO_ARTIFACTS
      Description: Build Download Docker container
      EncryptionKey:
        Ref: EncryptionKey
      Environment:
        ComputeType: BUILD_GENERAL1_MEDIUM
        EnvironmentVariables:
        - Name: IMAGE_TAG
          Value: latest
        - Name: IMAGE_REPO_NAME
          Value:
            Ref: DownloadContainerRegistry
        - Name: ACCOUNT_ID
          Value:
            Ref: AWS::AccountId
        - Name: BUILD_CONTEXT
          Value: infrastructure/docker/download
        - Name: WAIT_CONDITION_HANDLE
          Value:
            Ref: CodeBuildWaitHandle
        Image: aws/codebuild/standard:6.0
        ImagePullCredentialsType: CODEBUILD
        PrivilegedMode: true
        Type: LINUX_CONTAINER
      ResourceAccessRole:
        Fn::GetAtt:
        - CodeBuildRole
        - Arn
      ServiceRole:
        Fn::GetAtt:
        - CodeBuildRole
        - Arn
      Source:
        BuildSpec: infrastructure/docker/download/buildspec_with_wait.yaml
        GitCloneDepth: 1
        Location:
          Fn::GetAtt:
          - CodeRepository
          - CloneUrlHttp
        Type: CODECOMMIT
      SourceVersion: refs/heads/main
      Tags:
      - Key: Application
        Value:
          Ref: ApplicationName
      - Key: StackId
        Value:
          Ref: AWS::StackId
      - Key: SolutionId
        Value: SO9164
  StartDownloadContainerBuild:
    Type: Custom::ResourceForBuildingContainer
    Properties:
      ServiceToken:
        Fn::GetAtt: StartCodeBuildLambda.Arn
      ProjectName:
        Ref: CodeBuildProjectDownload
      Timestamp:
        Ref: Timestamp
  CodeBuildWaitHandle:
    Type: AWS::CloudFormation::WaitConditionHandle
  CodeBuildWaitCondition:
    Type: AWS::CloudFormation::WaitCondition
    Condition: DownloadFsxDataCondition
    DependsOn:
    - CodeBuildProjectDownload
    Properties:
      Handle:
        Ref: CodeBuildWaitHandle
      Timeout: 600
      Count: 1
  DeleteDownloadContainerRepo:
    Type: Custom::ResourceForDeletingECR
    Properties:
      ServiceToken:
        Fn::GetAtt: DeleteContainerRepoLambda.Arn
      ContainerRepo:
        Ref: DownloadContainerRegistry
Outputs:
  CodeRepositoryURI:
    Description: Code repository URI
    Value:
      Fn::GetAtt: CodeRepository.CloneUrlHttp
  CodeBuildEncryptionKey:
    Description: Encryption key for CodeBuild jobs
    Value:
      Ref: EncryptionKey
  CodeBuildRoleArn:
    Description: IAM role for CodeBuild jobs
    Value:
      Fn::GetAtt: CodeBuildRole.Arn
  StartCodeBuildLambdaArn:
    Description: Lambda function for building container repos
    Value:
      Fn::GetAtt: StartCodeBuildLambda.Arn
  DeleteContainerRepoLambdaArn:
    Description: Lambda function for deleting container repos
    Value:
      Fn::GetAtt: DeleteContainerRepoLambda.Arn
  DownloadContainerRegistryURI:
    Description: URI of the download container
    Value:
      Fn::GetAtt: DownloadContainerRegistry.RepositoryUri
  DownloadContainerRegistryName:
    Description: Name of the download container repository
    Value:
      Ref: DownloadContainerRegistry

[brianloyal]

Hmm, looks like I missed a reference in the last update. Thanks for catching it, @ejjordan - we'll get that fixed today or tomorrow

[brianloyal]

Hey @ejjordan can you verify that you're running the most recent version of the code? I don't see some of the updates I made 2 weeks ago in the code you pasted above.

[ejjordan]

Sorry for taking so long to respond to this! I hadn't updated my email preferences since switching jobs so the notification went somewhere I wasn't checking regularly. I just used the create stack button, so perhaps the issue is there. I am on vacation for the next week, but plan to try this again when I am back the week of the 21st.