(aws-cdk-lib/aws-glue): CfnSchema Rate Exceeded Error

[Borduhh]

Describe the bug

When destroying and redeploying our stack with 75 AWS Glue Schemas, we get the following error:

Unable to retrieve Arn attribute for AWS::Glue::Registry, with error message Error occurred during operation 'Rate exceeded (Service: Glue, Status Code: 400, Request ID: beb746c9-841c-4c6b-b5b0-04314602d889)

However, it works as expected if we wait some time and then deploy the stack.

Expected Behavior

The stack deploys successfully.

Current Behavior

The stack fails to deploy with a Rate Exceeded Error.

Reproduction Steps

1. Create a stack with 75 schemas

import * as cdk from 'aws-cdk-lib';
import { CfnRegistry, CfnSchema } from 'aws-cdk-lib/aws-glue';
import { Construct } from 'constructs';
import { readFileSync } from 'fs';
import { join } from 'path';

export class CdkSchmeaTestStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const registry = new CfnRegistry(this, 'TestRegistry', {
      name: 'Test-Registry',
    });

    const attrArn = registry.attrArn;

    const schmeas = Array(75)
      .fill('')
      .map(
        (s, index) =>
          new CfnSchema(this, `Schema${index}`, {
            compatibility: 'FORWARD',
            dataFormat: 'JSON',
            registry: {
              arn: attrArn,
            },
            name: `Schema${index}`,
            schemaDefinition: "{"$schema":"http://json-schema.org/draft-07/schema","title":"Test","type":"object","properties":{"testField":{"type":"string"}},"required":["testField"]}",
          })
      );
  }
}

2. Delete that stack
3. Try to redeploy that stack

Possible Solution

1. Batch the creation of schemas.
2. Increase the Rate Limit. This is being advertised as a competitor to Confluent Schema Registry, so it is not uncommon to have hundreds of schemas that need to be updated for a Kafka instance.

Additional Information/Context

No response

CDK CLI Version

2.39.1 (build f188fac)

Framework Version

No response

Node.js Version

v16.15.1

OS

macOS 12.6 (21G115)

Language

Typescript

Language Version

3.9.10

Other information

No response

[peterwoodworth]

Increase the Rate Limit. This is being advertised as a competitor to Confluent Schema Registry, so it is not uncommon to have hundreds of schemas that need to be updated for a Kafka instance.

This would be a request for Glue as a service. I'm not sure if you're able to request for an increase here for your account individually, but if you can I recommend doing that

Batch the creation of schemas.

This should honestly be done by CloudFormation automatically, but that would be out of our control as well.

A way other customers have worked around similar issues is by creating dependencies from CfnSchemas to other CfnSchemas, slowing down the deployment. I'm going to convert this issue to a discussion so that more discussion on this workaround can take place