Creating A Custom ParameterGroup For RDS ServerlessCluster - Any Better Options?

[AjkayAlan]

Recently I have been using Amazon Aurora PostgreSQL Serverless. As I have spun up more instances, I have recognized that there are some parameters in the parameter group that I want to adjust (for instance, rds.force_ssl). Based on Amazon's Guidance, it is suggested that I create a parameter group for my cluster based on a default parameter group.

This turns out to be a really weird and tricky thing to do "well" with CDK (specifically with Serverless). Basically, there are a number of interesting hurdles:

1. ParameterGroup.fromParameterGroupName only imports an existing parameter group - it does not allow you to mutate or make a copy of the parameter group. So, I cant get away with something like the following (in Python):

   parameter_group = rds.ParameterGroup.from_parameter_group_name(
       self, "ParameterGroup", "default.aurora-postgresql10"
   )
   parameter_group.add_parameter("rds.force_ssl", "1")

2. If you create a new ParameterGroup, you can't just keep the engine version vague like you can in the ServerlessCluster construct. So I can't do something like the following (in Python):

   parameter_group = rds.ParameterGroup(
       self,
       f"{construct_id}-ParameterGroup",
       engine=rds.DatabaseClusterEngine.AURORA_POSTGRESQL,
       parameters={
           "rds.force_ssl": "1"
       }
   )

   This will throw an error: jsii.errors.JSIIError: ParameterGroup cannot be used with an engine that doesn't specify a version

The ONLY way I could figure out how to get this to work was by building a parameter group using a specific AuroraPostgresEngineVersion

Here is an example in Python:

parameter_group = rds.ParameterGroup(
    self,
    "ParameterGroup",
    engine=rds.DatabaseClusterEngine.aurora_postgres(
        version=rds.AuroraPostgresEngineVersion.VER_10_14
    ),
    parameters={
        "rds.force_ssl": "1"
    }
)

From there, I can create a ServerlessCluster:

cluster = rds.ServerlessCluster(
    self,
    "Database",
    engine=rds.DatabaseClusterEngine.AURORA_POSTGRESQL,
    vpc=database_stack_props["vpc"],
    backup_retention=cdk.Duration.days(7),
    default_database_name="postgres",
    enable_data_api=True,
    parameter_group=parameter_group,
    removal_policy=cdk.RemovalPolicy.SNAPSHOT,
    vpc_subnets=ec2.SubnetSelection(subnet_type=ec2.SubnetType.ISOLATED),
)

Overall, it feels a bit weird to specify a specific version, because I don't actually know what version Amazon Aurora PostgreSQL Serverless is using behind the scenes. Has anyone else solved this in a more graceful way?

If not, I think there could be a few ways to make this a bit nicer from a CDK perspective:

1. Add a static method to ParameterGroup, something like createFromParameterGroupName, which effectively takes a copy and makes it your own, so you can do number 1 above
2. Allow ParameterGroup to be defined without a specific version (though that may be breaking)
3. Introduce a new enum for rds.DatabaseClusterEngine named AURORA_POSTGRESQL_SERVERLESS (and add respective serverless enums for other database types), which do not require a specific version to be specified (or magically pull that version behind the scenes).

Are there any other better options or does anyone else have any good ideas?

[skinny85]

Hey @AjkayAlan ,

this is actually an RDS limitation. While you can create a Serverless PostgreSQL cluster without providing it a specific version, you cannot create a PostgreSQL ParameterGroup that doesn't specify a (major) version.

We strongly advise providing a version also when creating the Cluster. Not providing it is actually an availability risk, like we say in the documentation of DatabaseClusterEngine.AURORA_POSTGRESQL, if RDS changes the major version of PostgreSQL that they consider the default on the service side, and your code breaks with the new major version.

Thanks,
Adam

[AjkayAlan]

Totally get that @skinny85, and it makes sense that we provide a version. So where I am stuck - just looking at CDK docs/enums, I can't really tell what versions are supported by Serverless.

The only way I could tell is navigating into the console, attempting to create a Serverless database, and then seeing what tooltips are out there:

Alternatively, I see that it is specified in the AWS docs:

My concern with Serverless is that if a new version is introduced for Serverless (lets say Serverless is bumped to v10.14), my code is pointed directly to 10.12, which then is no longer compatible with Serverless, correct?

It feels like a weird state - for traditional RDS, absolutely I should be pinning a version (since I am in total control). However, for Serverless, I am somewhat at the mercy of what AWS decides is "the" supported version, which may break my code down the line.

Am I correct in my thinking that if I pin a specific version, sometime in the future Serverless may not support that version?

[skinny85]

Hmm, you're right @AjkayAlan . I was actually under the impression that more versions supported Serverless than just 10.12, but apparently I was wrong.

Can you open us an issue about this? We'll have to think how to best tackle this on our end.

Am I correct in my thinking that if I pin a specific version, sometime in the future Serverless may not support that version?

This is the risk you run always with RDS. RDS constantly changes the supported versions of their databases. The difference is that if you don't pin to a specific version, RDS will do this migration for you, under the scenes, without you knowing about it. If you pin, you're in control when this migration happens.

[AjkayAlan]

Ok, I have #14031 submitted. Thanks for the discussion!

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.