updated to support Claude sonnet 4.0 and inference profiles (#32)

* updated to support Claude sonnet 4.0 and inference profiles

* fix: AMI version prompt

* fix: added impact analysis

* fix: version bump, docs update

Author: quixoticmonk

CHANGELOG.md

@@ -0,0 +1,17 @@
+# Changelog
+
+## [Unreleased]
+
+## [1.0.0] - 2025-10-03
+
+### Added
+- Enhanced AI analysis with structured impact assessment including security concerns, configuration issues, operational impact, and recommendations
+- Improved error handling and JSON parsing for Bedrock responses
+- Better AMI analysis with direct technical output
+
+### Changed
+- **BREAKING**: Updated default Bedrock model from `anthropic.claude-3-sonnet-20240229-v1:0` to `global.anthropic.claude-sonnet-4-20250514-v1:0` (supports cross-region inference profiles)
+- Increased default Lambda timeout from 120 seconds to 300 seconds for better performance with Claude 4.0
+- Restructured AI prompts for more focused and technical analysis output
+- Improved system prompts to reduce conversational language in responses
+- Enhanced JSON response parsing with better error handling and fallback mechanisms

README.md

@@ -199,14 +199,14 @@ Enhance your Terraform workflows with AI-powered insights while maintaining secu
 |------|-------------|------|---------|:--------:|
 | <a name="input_aws_region"></a> [aws\_region](#input\_aws\_region) | The region from which this module will be executed. | `string` | n/a | yes |
 | <a name="input_hcp_tf_org"></a> [hcp\_tf\_org](#input\_hcp\_tf\_org) | HCP Terraform Organization name | `string` | n/a | yes |
-| <a name="input_bedrock_llm_model"></a> [bedrock\_llm\_model](#input\_bedrock\_llm\_model) | Bedrock LLM model to use | `string` | `"anthropic.claude-3-sonnet-20240229-v1:0"` | no |
+| <a name="input_bedrock_llm_model"></a> [bedrock\_llm\_model](#input\_bedrock\_llm\_model) | Bedrock LLM model to use (supports cross-region inference profiles) | `string` | `"global.anthropic.claude-sonnet-4-20250514-v1:0"` | no |
 | <a name="input_cloudwatch_log_group_name"></a> [cloudwatch\_log\_group\_name](#input\_cloudwatch\_log\_group\_name) | RunTask CloudWatch log group name | `string` | `"/hashicorp/terraform/runtask/"` | no |
 | <a name="input_cloudwatch_log_group_retention"></a> [cloudwatch\_log\_group\_retention](#input\_cloudwatch\_log\_group\_retention) | Lambda CloudWatch log group retention period | `string` | `"365"` | no |
 | <a name="input_deploy_waf"></a> [deploy\_waf](#input\_deploy\_waf) | Set to true to deploy CloudFront and WAF in front of the Lambda function URL | `string` | `false` | no |
 | <a name="input_event_bus_name"></a> [event\_bus\_name](#input\_event\_bus\_name) | EventBridge event bus name | `string` | `"default"` | no |
 | <a name="input_event_source"></a> [event\_source](#input\_event\_source) | EventBridge source name | `string` | `"app.terraform.io"` | no |
 | <a name="input_lambda_architecture"></a> [lambda\_architecture](#input\_lambda\_architecture) | Lambda architecture (arm64 or x86\_64) | `string` | `"x86_64"` | no |
-| <a name="input_lambda_default_timeout"></a> [lambda\_default\_timeout](#input\_lambda\_default\_timeout) | Lambda default timeout in seconds | `number` | `120` | no |
+| <a name="input_lambda_default_timeout"></a> [lambda\_default\_timeout](#input\_lambda\_default\_timeout) | Lambda default timeout in seconds | `number` | `300` | no |
 | <a name="input_lambda_python_runtime"></a> [lambda\_python\_runtime](#input\_lambda\_python\_runtime) | Lambda Python runtime | `string` | `"python3.11"` | no |
 | <a name="input_lambda_reserved_concurrency"></a> [lambda\_reserved\_concurrency](#input\_lambda\_reserved\_concurrency) | Maximum Lambda reserved concurrency, make sure your AWS quota is sufficient | `number` | `10` | no |
 | <a name="input_name_prefix"></a> [name\_prefix](#input\_name\_prefix) | Name to be used on all the resources as identifier. | `string` | `"runtask-tf-plan-analyzer"` | no |

VERSION

@@ -1 +1 @@
-v0.2.0
+v1.0.0

lambda/runtask_fulfillment/ai.py

@@ -34,40 +34,16 @@ def eval(tf_plan_json):
 
     logger.info("##### Evaluating Terraform plan output #####")
     prompt = """
-    List the resources that will be created, modified or deleted in the following terraform plan using the following rules:
-    1. Think step by step using the "thinking" json field
-    2. For AMI changes, include the old and new AMI ID
-    3. Use the following schema. Skip the preamble:
-    <schema>
-    {
-        "$id": "https://example.com/arrays.schema.json",
-        "$schema": "https://json-schema.org/draft/2020-12/schema",
-        "type": "object",
-        "properties": {
-            "thinking": {
-                "type": "string",
-                "description": "Think step by step"
-            },
-            "resources": {
-                "type": "string",
-                "description": "A list of resources that will be created, modified or deleted"
-            }
-        }
-    }
-    </schema>
-    Here is an example of the output:
-        <example>
-        {
-        "thinking": "To list the resources that will be created, modified or deleted, I will go through the terraform plan and look for the 'actions' field in each resource change. If the actions include 'create', 'update', or 'delete', I will add that resource to the list. For AMI changes, I will include the old and new AMI ID.",
-        "resources": "The following resources will be modified: RESOURCES"
-        }
-        </example>
-    Now, list the resources that will be created, modified or deleted in the following terraform plan"""
+    You must respond with ONLY a JSON object. Do not include any explanatory text, conversation, or markdown formatting.
+
+    Analyze the terraform plan and return this exact JSON structure:
+    {"thinking": "brief analysis", "resources": "list of resources being created, modified, or deleted", "impact_analysis": "assessment formatted as markdown with sections: ## 🔍 Impact Analysis\n\n### 🚨 Security Concerns\n- **Critical/High/Medium**: Description\n- **Risk Level**: Assessment\n\n### ⚠️ Configuration Issues\n- **Issue Type**: Description\n- **Impact**: Consequence\n\n### 📊 Operational Impact\n- **Infrastructure**: What's being deployed\n- **Cost**: Cost implications\n\n### 💡 Recommendations\n- **Priority 1**: Most critical fix\n- **Priority 2**: Secondary concerns\n- **Warning**: Important warnings"}
+
+    Terraform plan:
+    """
 
     prompt += f"""
-    <terraform_plan>
     {tf_plan_json["resource_changes"]}
-    </terraform_plan>
     """
 
     messages = [
@@ -89,54 +65,34 @@ def eval(tf_plan_json):
 
     logger.debug("Analysis response: {}".format(analysis_response))
 
-    analysis_response_text= clean_response(analysis_response["content"][0]["text"])["resources"]
+    try:
+        parsed_response = clean_response(analysis_response["content"][0]["text"])
+        analysis_response_text = parsed_response["resources"]
+        impact_analysis_text = parsed_response.get("impact_analysis", "No impact analysis available")
+    except Exception as e:
+        logger.error(f"Error parsing analysis response: {e}")
+        analysis_response_text = "Error: Could not parse Terraform plan analysis"
+        impact_analysis_text = "Error: Could not parse impact analysis"
 
     logger.debug("Analysis response Text: {}".format(analysis_response_text))
 
     #####################################################################
     ######## Secondly, evaluate AMIs per analysis                ########
     #####################################################################
     logger.info("##### Evaluating AMI information #####")
-    prompt = """
-        Find additional details of infrastructure changes using the following rules
-        1. For Amazon machine image (AMI or image_id) modifications, compare the old AMI information against the new AMI, including linux kernel, docker and ecs agent using the get_ami_releases function.
-        2. Think step by step using "thinking" tags field
-        3. Use the following schema. Skip the preamble:
-        <output>
-        <thinking>
-        </thinking>
-        <result>
-            ## Current AMI ID
-                * AMI name:
-                * OS Architecture:
-                * OS Name:
-                * kernel:
-                * docker version:
-                * ECS agent:
-
-            ## New AMI ID
-                * AMI name:
-                * kernel:
-                * OS Architecture:
-                * OS Name:
-                * docker version:
-                * ECS agent:
-        </result>
-        <output>
-        Now, given the following analysis, compare any old with new AMIs:
-        """
+    prompt = f"""
+    For any Amazon Machine Image (AMI) changes in this analysis, use the get_ami_releases function to compare old and new AMI details including kernel, docker, and ECS agent versions.
 
-    prompt += f"""
-        <analysis>{analysis_response_text}</analysis>
-        """
+    Analysis: {analysis_response_text}
+    """
 
     messages = [{"role": "user", "content": [{"text": prompt}]}]
 
     stop_reason, response = stream_messages(
         bedrock_client=bedrock_client,
         model_id=model_id,
         messages=messages,
-        system_text=system_text,
+        system_text="Provide direct, technical analysis of AMI changes without conversational language.",
         tool_config=tool_config,
     )
 
@@ -173,53 +129,51 @@ def eval(tf_plan_json):
             bedrock_client=bedrock_client,
             model_id=model_id,
             messages=messages,
-            system_text=system_text,
+            system_text="Provide direct, technical analysis of AMI changes without conversational language.",
             tool_config=tool_config,
-            stop_sequences=["</result>"],
         )
 
         # Add response to message history
         messages.append(response)
 
-    # Try to parse output as XML and look for the <output> tag
-    try:
-        root = ET.fromstring(response["content"][0]["text"])
-        result = root.find("result").text
-        logger.info("Parsed : {}".format(result))
-    except Exception as e:
+    # Extract the actual response text from Bedrock
+    if response and "content" in response and len(response["content"]) > 0:
         result = response["content"][0]["text"]
-        logger.info("Non Parsed : {}".format(result))
+        logger.debug("AMI analysis response: {}".format(result))
+    else:
+        result = "Error: No AMI analysis response received from Bedrock"
+        logger.error("No AMI analysis content received from Bedrock")
 
     #####################################################################
     ######### Third, generate short summary                     #########
     #####################################################################
 
     logger.info("##### Generating short summary #####")
     prompt = f"""
-        List the resources that will be created, modified or deleted in the following terraform plan using the following rules:
-        - Provide summary of the infrastructure changes
-        - Highlight major components of the changes such as what Terraform modules is executed
-        - Summarize what each Terraform modules does
-        - Highlight any resources that being replaced or deleted
-        - Highlight any outputs if available
-
-        <terraform_plan>
-        {tf_plan_json["resource_changes"]}
-        </terraform_plan>
-        """
+    Provide a concise summary of these Terraform changes. Focus on what resources are being created, modified, or deleted:
+
+    {tf_plan_json["resource_changes"]}
+    """
     message_desc = [{"role": "user", "content": [{"text": prompt}]}]
     stop_reason, response = stream_messages(
         bedrock_client=bedrock_client,
         model_id=model_id,
         messages=message_desc,
-        system_text=system_text,
-        tool_config=tool_config,
-        stop_sequences=["</result>"],
+        system_text="Provide a direct, technical summary without conversational language.",
+        tool_config=None,
     )
-    description = response["content"][0]["text"]
+
+    # Extract the actual response text from Bedrock
+    if response and "content" in response and len(response["content"]) > 0:
+        description = response["content"][0]["text"]
+        logger.debug("Full Bedrock response: {}".format(description))
+    else:
+        description = "Error: No response received from Bedrock"
+        logger.error("No response content received from Bedrock")
 
     logger.info("##### Report #####")
     logger.info("Analysis : {}".format(analysis_response_text))
+    logger.info("Impact Analysis: {}".format(impact_analysis_text))
     logger.info("AMI summary: {}".format(result))
     logger.info("Terraform plan summary: {}".format(description))
 
@@ -232,6 +186,12 @@ def eval(tf_plan_json):
         results.append(generate_runtask_result(outcome_id="Plan-Summary", description="Summary of Terraform plan", result="Output omitted due to : {}".format(guardrail_response)))
         description = "Bedrock guardrail triggered : {}".format(guardrail_response)
 
+    guardrail_status, guardrail_response = guardrail_inspection(str(impact_analysis_text))
+    if guardrail_status:
+        results.append(generate_runtask_result(outcome_id="Impact-Analysis", description="Security and operational impact assessment", result=impact_analysis_text[:9000]))
+    else:
+        results.append(generate_runtask_result(outcome_id="Impact-Analysis", description="Security and operational impact assessment", result="Output omitted due to : {}".format(guardrail_response)))
+
     guardrail_status, guardrail_response = guardrail_inspection(str(result))
     if guardrail_status:
         results.append(generate_runtask_result(outcome_id="AMI-Summary", description="Summary of AMI changes", result=result[:700]))
@@ -279,8 +239,28 @@ def guardrail_inspection(input_text, input_mode = 'OUTPUT'):
         return True, "Guardrail inspection skipped"
 
 def clean_response(json_str):
-    # Remove any tags in the format <tag> or </tag>
-    cleaned_str = re.sub(r'<\/?[\w\s]+>', '', json_str)
-    last_brace_index = cleaned_str.rfind('}')
-    cleaned_str = cleaned_str[:last_brace_index + 1]
-    return json.loads(cleaned_str)
+    try:
+        # First try to parse as-is
+        return json.loads(json_str)
+    except json.JSONDecodeError:
+        try:
+            # Remove any tags in the format <tag> or </tag>
+            cleaned_str = re.sub(r'<\/?[\w\s]+>', '', json_str)
+
+            # Find JSON content between braces
+            start_brace = cleaned_str.find('{')
+            last_brace = cleaned_str.rfind('}')
+
+            if start_brace != -1 and last_brace != -1 and last_brace > start_brace:
+                json_content = cleaned_str[start_brace:last_brace + 1]
+                return json.loads(json_content)
+            else:
+                logger.error(f"No valid JSON braces found in response: {json_str}")
+                return {"resources": "Error: No valid JSON structure found"}
+
+        except json.JSONDecodeError as e:
+            logger.error(f"JSON decode error after cleaning: {e}, Original string: {json_str}")
+            return {"resources": "Error: Could not parse response as JSON"}
+    except Exception as e:
+        logger.error(f"Unexpected error in clean_response: {e}, Original string: {json_str}")
+        return {"resources": "Error: Unexpected parsing error"}

variables.tf

@@ -89,7 +89,7 @@ variable "lambda_reserved_concurrency" {
 variable "lambda_default_timeout" {
   description = "Lambda default timeout in seconds"
   type        = number
-  default     = 120
+  default     = 300
 }
 
 variable "lambda_architecture" {
@@ -148,7 +148,7 @@ variable "waf_managed_rule_set" {
 }
 
 variable "bedrock_llm_model" {
-  description = "Bedrock LLM model to use"
+  description = "Bedrock LLM model to use (supports cross-region inference profiles)"
   type        = string
-  default     = "anthropic.claude-3-sonnet-20240229-v1:0"
+  default     = "global.anthropic.claude-sonnet-4-20250514-v1:0"
 }