Add configuration-based assessment and summarization control with runtime enable/disable

Author: Bob Strahan

CHANGELOG.md

@@ -13,14 +13,29 @@ SPDX-License-Identifier: MIT-0
   - **Cost Optimization**: When disabled, no LLM API calls or S3 operations are performed
   - **Configuration Example**: Set `summarization.enabled: false` to disable, `enabled: true` to enable (default)
 
-### Changed
-- **State Machine Simplification**: Removed `SummarizationChoice` conditional states from all patterns (Pattern 1, 2, 3) for cleaner workflows
-- **Service Logic Enhancement**: SummarizationService now checks configuration `enabled` flag at the beginning of `process_document()`
-- **Configuration Schema Updates**: Added `enabled` boolean property to summarization sections in all CloudFormation template schemas
+- **Configuration-Based Assessment Control**
+  - Assessment can now be enabled/disabled via configuration file `assessment.enabled` property instead of CloudFormation stack parameter
+  - **Key Benefits**: Runtime control without stack redeployment, zero LLM costs when disabled, simplified state machine architecture, backward compatible defaults
+  - **Implementation**: Always calls AssessmentStep but service skips processing when `enabled: false`
+  - **Cost Optimization**: When disabled, no LLM API calls or S3 operations are performed
+  - **Configuration Example**: Set `assessment.enabled: false` to disable, `enabled: true` to enable (default)
+
+### Changed  
+- **State Machine Simplification**: Removed `SummarizationChoice` and `AssessmentChoice` conditional states from all patterns (Pattern 2, 3) for cleaner workflows
+- **Service Logic Enhancement**: SummarizationService and AssessmentService now check configuration `enabled` flag at the beginning of processing methods
+- **Configuration Schema Updates**: Added `enabled` boolean property to summarization and assessment sections in all CloudFormation template schemas
+- Updated all sample configurations to include `summarization.enabled: true` and `assessment.enabled: true`
+- Updated configuration documentation with new summarization and assessment control approaches
 
 ### Removed
-- **CloudFormation Parameter**: Removed `IsSummarizationEnabled` parameter from all pattern templates (patterns/pattern-1, pattern-2, pattern-3)
-- **Related Conditions**: Removed `IsSummarizationEnabled` conditions and state machine definition substitutions
+- **CloudFormation Parameters**: Removed `IsSummarizationEnabled` and `IsAssessmentEnabled` parameters from all pattern templates
+- **Related Conditions**: Removed parameter conditions and state machine definition substitutions for both features
+- **Conditional Logic**: Eliminated complex conditional logic from state machine definitions for summarization and assessment steps
+
+### Fixed
+- **CloudFormation Template Deployment Error**: Fixed "Template format error: Unresolved resource dependencies [IsAssessmentEnabled]" by removing final parameter reference in main template PATTERN3STACK parameters
+- **State Machine Logic**: Simplified conditional assessment/summarization steps that were causing complex workflow logic
+- **Parameter Dependencies**: Cleaned up all CloudFormation parameter dependencies and references
 
 ### Documentation
 - **Updated Documentation**: Enhanced docs/configuration.md, docs/architecture.md, and all pattern-specific docs (pattern-1.md, pattern-2.md, pattern-3.md)

config_library/pattern-2/bank-statement-sample/config.yaml

@@ -369,6 +369,7 @@ summarization:
   system_prompt: >-
     You are a document summarization expert who can analyze and summarize documents from various domains including medical, financial, legal, and general business documents. Your task is to create a summary that captures the key information, main points, and important details from the document. Your output must be in valid JSON format. \nSummarization Style: Balanced\\nCreate a balanced summary that provides a moderate level of detail. Include the main points and key supporting information, while maintaining the document's overall structure. Aim for a comprehensive yet concise summary.\n Your output MUST be in valid JSON format with markdown content. You MUST strictly adhere to the output format specified in the instructions.
 assessment:
+  enabled: true
   image:
     target_height: ''
     target_width: ''

config_library/pattern-2/lending-package-sample/config.yaml

@@ -1144,6 +1144,7 @@ summarization:
   system_prompt: >-
     You are a document summarization expert who can analyze and summarize documents from various domains including medical, financial, legal, and general business documents. Your task is to create a summary that captures the key information, main points, and important details from the document. Your output must be in valid JSON format. \nSummarization Style: Balanced\\nCreate a balanced summary that provides a moderate level of detail. Include the main points and key supporting information, while maintaining the document's overall structure. Aim for a comprehensive yet concise summary.\n Your output MUST be in valid JSON format with markdown content. You MUST strictly adhere to the output format specified in the instructions.
 assessment:
+  enabled: true
   image:
     target_height: ''
     target_width: ''

config_library/pattern-2/rvl-cdip-package-sample-with-few-shot-examples/config.yaml

@@ -886,6 +886,7 @@ pricing:
       - name: cacheWriteInputTokens
         price: '3.75E-6'
 assessment:
+  enabled: true
   image:
     target_height: ''
     target_width: ''

config_library/pattern-2/rvl-cdip-package-sample/config.yaml

@@ -609,6 +609,7 @@ summarization:
   system_prompt: >-
     You are a document summarization expert who can analyze and summarize documents from various domains including medical, financial, legal, and general business documents. Your task is to create a summary that captures the key information, main points, and important details from the document. Your output must be in valid JSON format. \nSummarization Style: Balanced\\nCreate a balanced summary that provides a moderate level of detail. Include the main points and key supporting information, while maintaining the document's overall structure. Aim for a comprehensive yet concise summary.\n Your output MUST be in valid JSON format with markdown content. You MUST strictly adhere to the output format specified in the instructions.
 assessment:
+  enabled: true
   image:
     target_height: ''
     target_width: ''

config_library/pattern-3/rvl-cdip-package-sample/config.yaml

@@ -469,6 +469,7 @@ summarization:
   system_prompt: >-
     You are a document summarization expert who can analyze and summarize documents from various domains including medical, financial, legal, and general business documents. Your task is to create a summary that captures the key information, main points, and important details from the document. Your output must be in valid JSON format. \nSummarization Style: Balanced\\nCreate a balanced summary that provides a moderate level of detail. Include the main points and key supporting information, while maintaining the document's overall structure. Aim for a comprehensive yet concise summary.\n Your output MUST be in valid JSON format with markdown content. You MUST strictly adhere to the output format specified in the instructions.
 assessment:
+  enabled: true
   image:
     target_width: ''
     target_height: ''

docs/assessment.md

@@ -54,19 +54,33 @@ The assessment step is conditionally integrated into Pattern-2's ProcessSections
 
 ## Configuration
 
-### Deployment Parameter
+### Configuration-Based Control
 
-Enable assessment during stack deployment:
+Assessment can now be controlled via the configuration file rather than CloudFormation stack parameters. This provides more flexibility and eliminates the need for stack redeployment when changing assessment behavior.
 
+**Configuration-based Control (Recommended):**
 ```yaml
-Parameters:
-  IsAssessmentEnabled:
-    Type: String
-    Default: "false"
-    AllowedValues: ["true", "false"]
-    Description: Enable assessment functionality for extraction confidence evaluation
+assessment:
+  enabled: true  # Set to false to disable assessment
+  model: us.amazon.nova-lite-v1:0
+  temperature: 0.0
+  # ... other assessment settings
 ```
 
+**Key Benefits:**
+- **Runtime Control**: Enable/disable without stack redeployment  
+- **Cost Optimization**: Zero LLM costs when disabled (`enabled: false`)
+- **Simplified Architecture**: No conditional logic in state machines
+- **Backward Compatible**: Defaults to `enabled: true` when property is missing
+
+**Behavior When Disabled:**
+- Assessment lambda is still called (minimal overhead)
+- Service immediately returns with logging: "Assessment is disabled via configuration"
+- No LLM API calls or S3 operations are performed
+- Document processing continues to completion
+
+**Migration Note**: The previous `IsAssessmentEnabled` CloudFormation parameter has been removed in favor of this configuration-based approach.
+
 ### Assessment Configuration Section
 
 Add the assessment section to your configuration YAML:
@@ -808,7 +822,7 @@ The assessment feature implements several cost optimization techniques:
 
 1. **Text Confidence Data**: Uses condensed OCR confidence information instead of full raw OCR results (80-90% token reduction)
 2. **Conditional Image Processing**: Images only processed when `{DOCUMENT_IMAGE}` placeholder is present
-3. **Optional Deployment**: Assessment infrastructure only deployed when `IsAssessmentEnabled=true`
+3. **Configuration-Based Control**: Assessment can be enabled/disabled via configuration `enabled` property for flexible deployment
 4. **Efficient Prompting**: Optimized prompt templates minimize token usage while maintaining accuracy
 5. **Configurable Image Dimensions**: Adjust image resolution to balance assessment quality and processing costs
 6. **Granular Assessment with Caching**: For complex documents, use granular assessment with prompt caching for 60-80% cost reduction
@@ -889,7 +903,7 @@ ValueError: "Assessment prompt template formatting failed: missing required plac
 ### Common Issues
 
 1. **Assessment Not Running**
-   - Verify `IsAssessmentEnabled=true` in deployment
+   - Verify `assessment.enabled: true` in configuration file
    - Check state machine definition includes assessment step
    - Confirm assessment Lambda function deployed successfully
 

docs/configuration.md

@@ -56,6 +56,35 @@ summarization:
 
 **Migration Note**: The previous `IsSummarizationEnabled` CloudFormation parameter has been removed in favor of this configuration-based approach.
 
+## Assessment Configuration
+
+### Enable/Disable Assessment
+
+Similar to summarization, assessment can now be controlled via the configuration file rather than CloudFormation stack parameters. This provides more flexibility and eliminates the need for stack redeployment when changing assessment behavior.
+
+**Configuration-based Control (Recommended):**
+```yaml
+assessment:
+  enabled: true  # Set to false to disable assessment
+  model: us.amazon.nova-lite-v1:0
+  temperature: 0.0
+  # ... other assessment settings
+```
+
+**Key Benefits:**
+- **Runtime Control**: Enable/disable without stack redeployment
+- **Cost Optimization**: Zero LLM costs when disabled (`enabled: false`)
+- **Simplified Architecture**: No conditional logic in state machines
+- **Backward Compatible**: Defaults to `enabled: true` when property is missing
+
+**Behavior When Disabled:**
+- Assessment lambda is still called (minimal overhead)
+- Service immediately returns with logging: "Assessment is disabled via configuration"
+- No LLM API calls or S3 operations are performed
+- Document processing continues to completion
+
+**Migration Note**: The previous `IsAssessmentEnabled` CloudFormation parameter has been removed in favor of this configuration-based approach.
+
 ## Stack Parameters
 
 Key parameters that can be configured during CloudFormation deployment:

docs/evaluation.md

@@ -43,7 +43,7 @@ The framework supports multiple comparison methods:
 
 ## Assessment Confidence Integration
 
-The evaluation framework automatically integrates with the assessment feature to provide enhanced quality insights. When documents have been processed with assessment enabled, the evaluation reports include confidence scores alongside traditional accuracy metrics.
+The evaluation framework automatically integrates with the assessment feature to provide enhanced quality insights. When documents have been processed with assessment enabled via the configuration `assessment.enabled: true` property, the evaluation reports include confidence scores alongside traditional accuracy metrics.
 
 ### Confidence Score Display
 

docs/pattern-2.md

@@ -619,16 +619,32 @@ The assessment feature runs after successful extraction and provides:
 
 ### Enabling Assessment
 
-Assessment is controlled by the `IsAssessmentEnabled` deployment parameter:
+Assessment can now be controlled via the configuration file rather than CloudFormation stack parameters. This provides more flexibility and eliminates the need for stack redeployment when changing assessment behavior.
 
-```bash
-# Deploy with assessment enabled
-aws cloudformation deploy \
-  --template-file template.yaml \
-  --parameter-overrides IsAssessmentEnabled=true
+**Configuration-based Control (Recommended):**
+```yaml
+assessment:
+  enabled: true  # Set to false to disable assessment
+  model: us.amazon.nova-lite-v1:0
+  temperature: 0.0
+  # ... other assessment settings
 ```
 
-When enabled, the assessment step is conditionally added to the state machine workflow:
+**Key Benefits:**
+- **Runtime Control**: Enable/disable without stack redeployment
+- **Cost Optimization**: Zero LLM costs when disabled (`enabled: false`)
+- **Simplified Architecture**: No conditional logic in state machines
+- **Backward Compatible**: Defaults to `enabled: true` when property is missing
+
+**Behavior When Disabled:**
+- Assessment lambda is still called (minimal overhead)
+- Service immediately returns with logging: "Assessment is disabled via configuration"
+- No LLM API calls or S3 operations are performed
+- Document processing continues to completion
+
+**Migration Note**: The previous `IsAssessmentEnabled` CloudFormation parameter has been removed in favor of this configuration-based approach.
+
+The assessment step is always called in the state machine workflow, but the service itself handles the enablement decision:
 
 ```
 OCRStep → ClassificationStep → ProcessPageGroups (Map State):