Release 1.61.0 (to main) #3011

Release 1.61.0 (to main)

Author: aahung

.cfnlintrc.yaml

@@ -120,6 +120,7 @@ ignore_templates:
   - tests/translator/output/**/state_machine_with_schedule_dlq_retry_policy.json
   - tests/translator/output/**/globals_for_function.json  # RuntimeManagementConfig
   - tests/translator/output/**/function_with_runtime_config.json  # RuntimeManagementConfig
+  - tests/translator/output/**/managed_policies_minimal.json  # Intentionally has non-existent managed policy name
 ignore_checks:
   - E2531 # Deprecated runtime; not relevant for transform tests
   - W2531 # EOL runtime; not relevant for transform tests

.coveragerc

@@ -3,6 +3,7 @@ branch = True
 omit =
     # Schema tested by `make check-black`
     samtranslator/schema/*
+    samtranslator/internal/schema_source/*
 [report]
 exclude_lines =
     pragma: no cover

DEVELOPMENT_GUIDE.md

@@ -158,11 +158,16 @@ Integration tests are covered in detail in the [INTEGRATION_TESTS.md file](INTEG
 
 ## Development guidelines
 
-1. **Do not resolve [intrinsic functions](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference.html).** Adding [`AWS::LanguageExtensions`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-languageextension-transform.html) before the `AWS::Serverless-2016-10-31` transform resolves most of them (see https://github.com/aws/serverless-application-model/issues/2533). For new properties, use [`Property`](https://github.com/aws/serverless-application-model/blob/c5830b63857f52e540fec13b29f029458edc539a/samtranslator/model/__init__.py#L36-L45) or [`PassThroughProperty`](https://github.com/aws/serverless-application-model/blob/dd79f535500158baa8e367f081d6a12113497e45/samtranslator/model/__init__.py#L48-L56) instead of [`PropertyType`](https://github.com/aws/serverless-application-model/blob/c39c2807bbf327255de8abed8b8150b18c60f053/samtranslator/model/__init__.py#L13-L33).
+1. **Do not resolve [intrinsic functions](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference.html).** Adding [`AWS::LanguageExtensions`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-languageextension-transform.html) before the `AWS::Serverless-2016-10-31` transform resolves most of them (see https://github.com/aws/serverless-application-model/issues/2533).
 2. **Do not break backward compatibility.** A specific SAM template should always transform into the same CloudFormation template. A template that has previously deployed successfully should continue to do so. Do not change logical IDs. Add opt-in properties for breaking changes. There are some exceptions, such as changes that do not impact resources (e.g. [`Metadata`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/metadata-section-structure.html)) or abstractions that can by design change over time.
 3. **Stick as close as possible to the underlying CloudFormation properties.** This includes both property names and values. This ensures we can pass values to CloudFormation and let it handle any intrinsic functions. In some cases, it also allows us to pass all properties as-is to a resource, which means customers can always use the newest properties, and we don’t spend effort maintaining a duplicate set of properties.
 4. **Only validate what’s necessary.** Do not validate properties if they’re passed directly to the underlying CloudFormation resource.
 5. **Add [type hints](https://peps.python.org/pep-0484/) to new code.** Strict typing was enabled in https://github.com/aws/serverless-application-model/pull/2558 by sprinkling [`# type: ignore`](https://peps.python.org/pep-0484/#compatibility-with-other-uses-of-function-annotations) across the existing code. Don't do that for new code. Avoid `# type: ignore`s at all cost. Instead, add types to new functions, and ideally add types to existing code it uses as well.
+6. **Do not use [`PropertyType`](https://github.com/aws/serverless-application-model/blob/c39c2807bbf327255de8abed8b8150b18c60f053/samtranslator/model/__init__.py#L13-L33) for new [`Resource`](https://github.com/aws/serverless-application-model/blob/13604cd2d9671cd6e774e5bfd610a03d82a08d76/samtranslator/model/__init__.py#L68) properties.**
+
+   For new properties of SAM resources, use [`Property`](https://github.com/aws/serverless-application-model/blob/c5830b63857f52e540fec13b29f029458edc539a/samtranslator/model/__init__.py#L36-L45) or [`PassThroughProperty`](https://github.com/aws/serverless-application-model/blob/dd79f535500158baa8e367f081d6a12113497e45/samtranslator/model/__init__.py#L48-L56) instead of [`PropertyType`](https://github.com/aws/serverless-application-model/blob/c39c2807bbf327255de8abed8b8150b18c60f053/samtranslator/model/__init__.py#L13-L33). This avoids [sneaky bugs](https://github.com/aws/serverless-application-model/pull/2495#discussion_r976715242) and ensures valid templates do not cause transform failures.
+
+   For new properties of CloudFormation resources, use [`GeneratedProperty`](https://github.com/aws/serverless-application-model/blob/79452f69bc1fcf918b8625c2f9005c74ab874801/samtranslator/model/__init__.py#L74-L82). It performs no runtime validation, reducing the risk of valid values causing transform failures.
 
 Code conventions
 ----------------

MANIFEST.in

@@ -8,6 +8,8 @@ recursive-include samtranslator/validator/sam_schema *.json
 include samtranslator/policy_templates_data/policy_templates.json
 include samtranslator/policy_templates_data/schema.json
 include samtranslator/model/connector_profiles/profiles.json
+include samtranslator/internal/data/aws_managed_policies.json
+include samtranslator/internal/schema_source/sam-docs.json
 include README.md
 include THIRD_PARTY_LICENSES
 

Makefile

@@ -26,7 +26,7 @@ black:
 black-check:
 	# Checking latest schema was generated (run `make schema` if this fails)
 	mkdir -p .tmp
-	python -m schema_source.schema --sam-schema .tmp/sam.schema.json --cfn-schema schema_source/cloudformation.schema.json --unified-schema .tmp/schema.json
+	python -m samtranslator.internal.schema_source.schema --sam-schema .tmp/sam.schema.json --cfn-schema schema_source/cloudformation.schema.json --unified-schema .tmp/schema.json
 	diff -u schema_source/sam.schema.json .tmp/sam.schema.json
 	diff -u samtranslator/schema/schema.json .tmp/schema.json
 	black --check setup.py samtranslator tests integration bin schema_source
@@ -60,14 +60,14 @@ fetch-schema-data:
 
 update-schema-data:
 	# Parse docs
-	bin/parse_docs.py .tmp/aws-sam-developer-guide/doc_source > schema_source/docs.json
+	bin/parse_docs.py .tmp/aws-sam-developer-guide/doc_source > samtranslator/internal/schema_source/sam-docs.json
 	bin/parse_docs.py --cfn .tmp/aws-cloudformation-user-guide/doc_source > schema_source/cloudformation-docs.json
 
 	# Add CloudFormation docs to CloudFormation schema
 	python bin/add_docs_cfn_schema.py --schema .tmp/cloudformation.schema.json --docs schema_source/cloudformation-docs.json > schema_source/cloudformation.schema.json
 
 schema:
-	python -m schema_source.schema --sam-schema schema_source/sam.schema.json --cfn-schema schema_source/cloudformation.schema.json --unified-schema samtranslator/schema/schema.json
+	python -m samtranslator.internal.schema_source.schema --sam-schema schema_source/sam.schema.json --cfn-schema schema_source/cloudformation.schema.json --unified-schema samtranslator/schema/schema.json
 
 # Update all schema data and schemas
 schema-all: fetch-schema-data update-schema-data schema

bin/parse_docs.py

@@ -89,10 +89,10 @@ def main() -> None:
         if args.cfn and not re.match(r"^\w+::\w+::\w+( \w+)?$", title):
             continue
         page = title if args.cfn else path.stem
-        for name, description in parse(text):
+        for name, raw_description in parse(text):
             if page not in props:
                 props[page] = {}
-            description = remove_first_line(description)  # Remove property name; already in the schema title
+            description = remove_first_line(raw_description)  # Remove property name; already in the schema title
             description = fix_markdown_code_link(description)
             prefix = (
                 "https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/"

bin/sam-translate.py

@@ -106,7 +106,7 @@ def transform_template(input_file_path, output_file_path):  # type: ignore[no-un
         sam_template = yaml_parse(f)  # type: ignore[no-untyped-call]
 
     try:
-        cloud_formation_template = transform(sam_template, {}, ManagedPolicyLoader(iam_client))  # type: ignore[no-untyped-call]
+        cloud_formation_template = transform(sam_template, {}, ManagedPolicyLoader(iam_client))
         cloud_formation_template_prettified = json.dumps(cloud_formation_template, indent=1)
 
         with open(output_file_path, "w") as f:

docs/globals.rst

@@ -81,6 +81,7 @@ Currently, the following resources and properties are being supported:
       Auth:
       Name:
       DefinitionUri:
+      MergeDefinitions:
       CacheClusterEnabled:
       CacheClusterSize:
       Variables:

integration/combination/test_connectors.py

@@ -26,6 +26,7 @@ def tearDown(self):
 
     @parameterized.expand(
         [
+            ("combination/connector_appsync_to_table",),
             ("combination/connector_function_to_function",),
             ("combination/connector_restapi_to_function",),
             ("combination/connector_httpapi_to_function",),

integration/config/service_names.py

@@ -35,3 +35,4 @@
 EVENT_INVOKE_CONFIG = "EventInvokeConfig"
 EPHEMERAL_STORAGE = "EphemeralStorage"
 API_KEY = "ApiKey"
+APP_SYNC = "AppSync"