Unable to import backup from S3 bucket through Terraform

[JuanPolanco37]

Description

We intend to create an Aurora RDS cluster & import the backup (Percona XtraBackup) from an existing cluster (in EC2) using Terraform for automation.
We receive the following error when trying to import the Percona Xtrabackup from the S3 bucket:
Error: error creating RDS DB Cluster (XXX) IAM Role (arn:aws:iam::######:role/test-us00-s3-import) Association: InvalidParameterValue: The feature name s3Import is not valid for the engine Aurora MySQL.

We have tried to manually reproduce this step through the AWS Console and it succeeded without any errors.

Versions

• Terraform:
  Terraform v0.13.7
• Provider(s):
  Terraform v0.13.7

Expected behavior

We expect that the newly created instance has the data migrated from the backup stored in the S3 bucket

Actual behavior

Error:
Error: error creating RDS DB Cluster (test-us00-rds) IAM Role (arn:aws:iam:##########:role/test-us00-s3-import) Association: InvalidParameterValue: The feature name s3Import is not valid for the engine Aurora MySQL.

[bryantbiggs]

hi @JuanPolanco37 - do you have a reproduction of your configuration, what version MySQL are you using, etc.?

[JuanPolanco37]

hello @bryantbiggs , sorry for the delay.
The current version of MySQL we're using is : 5.7 (2.10.2)

[JuanPolanco37]

@bryantbiggs , and here is the snippet where we try to import backup from S3 bucket to our Aurora RDS:

  iam_roles = {
    s3_import = {
      role_arn     = aws_iam_role.s3_import.arn
      feature_name = "s3Import"
    }
  }
  # S3 import https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Integrating.LoadFromS3.html
  s3_import = {
    source_engine_version = "5.7.12"
    bucket_name           = format("%s-%s", var.environment, "aurora-import")
    ingestion_role        = aws_iam_role.s3_import.arn
    bucket_prefix         = var.bucket_prefix
  }

[JuanPolanco37]

this is our main module for the creation of the RDS Cluster:

  source = "terraform-aws-modules/rds-aurora/aws"
  instances = {
    db01 = {
      publicly_accessible = false
    }
    db02 = {
      publicly_accessible = true
    }
  }
  name                                  = format("%s-rds-test", var.environment)
  engine                                = "aurora-mysql"
  engine_version                        = "5.7.mysql_aurora.2.10.2"
  instance_class                        = "db.r5.2xlarge"
  performance_insights_enabled          = true
  performance_insights_retention_period = 7
  master_username                       = "sysdba"
  publicly_accessible                   = true
  vpc_id                                = "test-us00-default"
  db_subnet_group_name                  = "test-us00-db"
  allowed_security_groups               = ["test-us00-db"]
  storage_encrypted                     = true
  apply_immediately                     = false
  monitoring_interval                   = 10
  preferred_backup_window               = "03:00-05:00"
  db_parameter_group_name               = "default.aurora-mysql5.7"
  db_cluster_parameter_group_name       = "default.aurora-mysql5.7"
  kms_key_id                            = aws_kms_key.aurora_backup_key.arn
  enabled_cloudwatch_logs_exports       = ["audit", "error", "general", "slowquery"]
  create_security_group                 = false
  create_db_subnet_group                = false
  tags = merge({
    Name        = format("%s-rds-test", var.environment),
    Environment = "test",
    Terraform   = "true"
  }, var.default_tags)
  iam_roles = {
    s3_import = {
      role_arn     = aws_iam_role.s3_import.arn
      feature_name = "s3Import"
    }
  }
  # S3 import [https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Integrating.LoadFromS3.html ](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Integrating.LoadFromS3.html)
  s3_import = {
    source_engine_version = "5.7.12"
    bucket_name           = format("%s-%s", var.environment, "aurora-import")
    ingestion_role        = aws_iam_role.s3_import.arn
    bucket_prefix         = var.bucket_prefix
  }
  depends_on = [
    aws_iam_role.s3_import,aws_iam_role_policy.s3_import,module.import_s3_bucket
  ]
}

and our current Role has the following permissions:

---------------------------------------------------------------------------------------------------------
"Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:ListAllMyBuckets",
                "s3:GetBucketLocation"
            ],
---------------------------------------------------------------------------------------------------------```

[JuanPolanco37]

@bryantbiggs,
we received a response from AWS Support regarding this issue and they have responded that the feature_name = "s3Import" is not compatible with Aurora MySQL, so they suggested to specify an empty "" string for feature name while creating an RDS Instance with Restore from S3 through Terraform as there is no need of providing feature name for Aurora MySQL engine.
This worked for us and I think it would be important to double check if this is something to be needed to address in the terraform code.

Best regards,
Juan Polanco

[bryantbiggs]

thanks for the info @JuanPolanco37 - strange that this is no longer accepted since the mysql example we have has worked for quite some time. but now I can see that Aurora MySQL does not support any feature names aws rds describe-db-engine-versions --engine aurora-mysql --query 'DBEngineVersions[*].SupportedFeatureNames'

[github-actions]

This issue has been automatically marked as stale because it has been open 30 days
with no activity. Remove stale label or comment or this issue will be closed in 10 days

[github-actions]

This issue was automatically closed because of stale in 10 days

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.