Skip to content

Commit e343b42

Browse files
author
Bob Strahan
committed
Merge branch 'develop' v0.4.6
2 parents b532497 + ea2e438 commit e343b42

File tree

170 files changed

+20949
-4672
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

170 files changed

+20949
-4672
lines changed

‎.github/workflows/developer-tests.yml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,7 @@ jobs:
5959
6060
- name: Install Node.js and basedpyright
6161
run: |
62-
curl -fsSL https://deb.nodesource.com/setup_20.x | bash -
62+
curl -fsSL https://deb.nodesource.com/setup_22.x | bash -
6363
apt-get install -y nodejs
6464
npm install -g basedpyright
6565
@@ -104,6 +104,7 @@ jobs:
104104
with:
105105
files: lib/idp_common_pkg/test-reports/test-results.xml
106106
check_name: Test Results
107+
comment_mode: off # Disable PR comments to avoid permission issues on fork PRs
107108

108109
- name: Code Coverage Report
109110
uses: irongut/[email protected]

‎CHANGELOG.md‎

Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,156 @@ SPDX-License-Identifier: MIT-0
55

66
## [Unreleased]
77

8+
## [0.4.6]
9+
810
### Added
911

12+
- **New State-Of-The-Art LLM Model Support**
13+
- Added support for Amazon Nova 2 Lite model (`us.amazon.nova-2-lite-v1:0`, `eu.amazon.nova-2-lite-v1:0`)
14+
- Added support for Claude Opus 4.5 model (`us.anthropic.claude-opus-4-5-20251101-v1:0`, `eu.anthropic.claude-opus-4-5-20251101-v1:0`)
15+
- Added support for Qwen 3 VL model (`qwen.qwen3-vl-235b-a22b`)
16+
- Available for configuration across all document processing steps
17+
18+
- **Test Studio for Comprehensive Test Management and Analysis**
19+
- Added unified web interface for managing test sets, running tests, and analyzing results directly from the UI
20+
- **Test Sets Tab**: Create and manage reusable test collections with three creation methods:
21+
- Pattern-based creation with file patterns to match existing data sets (Input Bucket and Test Set Bucket)
22+
- Zip upload with automatic extraction of `input/` and `baseline/` folder structure
23+
- **Test Executions Tab**: Unified interface combining test execution and results management:
24+
- Real-time status monitoring
25+
- Multi-select comparison for side-by-side test analysis
26+
- Integrated export and delete operations
27+
- **Key Features**: File structure validation, progress-aware status updates, cached metrics for improved performance, dual bucket support for flexible test organization
28+
- **Documentation**: Guide in `docs/test-studio.md` with architecture details and workflow examples
29+
30+
- **MCP Integration for External Application Access**
31+
- Added MCP (Model Context Protocol) integration enabling external applications (like Amazon Quick Suite) to access IDP analytics through AWS Bedrock AgentCore Gateway with secure OAuth 2.0 authentication
32+
- Implemented Analytics Agent with `search_genaiidp` tool for natural language queries of processed document data (statistics, trends, confidence scores, processing status)
33+
- Controlled by `EnableMCP` parameter (default: true); provides MCPServerEndpoint and authentication outputs for external application integration; documentation in `docs/mcp-integration.md`
34+
35+
- **Configurable Section Splitting Strategies for Enhanced Document Segmentation Control**
36+
- Added new `sectionSplitting` configuration option to control how classified pages are grouped into document sections
37+
- **Three Strategies Available**:
38+
- `disabled`: Entire document treated as single section with first detected class (simplest case)
39+
- `page`: One section per page preventing automatic joining of same-type documents (deterministic, solves Issue #146)
40+
- `llm_determined`: Uses LLM boundary detection with "Start"/"Continue" indicators (default, maintains existing behavior)
41+
- **Key Benefits**: Deterministic splitting for long documents with multiple same-type forms (e.g., multiple W-2s, multiple invoices), eliminates LLM boundary detection failures for critical government form processing, provides flexibility across simple to complex document scenarios
42+
- Resolves #146
43+
44+
### Changed
45+
46+
- **Improved Temperature and Top_P Parameter Logic for Deterministic Output**
47+
- Changed inference parameter selection logic to allow `temperature=0.0` for deterministic output (recommended by Anthropic and other model providers)
48+
- **New Logic**: Uses `top_p` only when it has a positive value (> 0); otherwise uses `temperature` including `temperature=0.0`
49+
- **Previous Logic**: Used `top_p` whenever `temperature=0.0`, preventing proper deterministic configuration
50+
- **Key Benefits**: Enables proper deterministic output with `temperature=0.0`, more intuitive parameter behavior, aligns with model provider best practices (Anthropic recommends `temperature=0` for consistent outputs)
51+
- **Affected Components**: Bedrock client (`lib/idp_common_pkg/idp_common/bedrock/client.py`), Agentic extraction service (`lib/idp_common_pkg/idp_common/extraction/agentic_idp.py`)
52+
- **Configuration Guidance**: Set `top_p: 0` to use `temperature` parameter; set `top_p` to positive value to override temperature
53+
- Set temperature to 0.0 in discovery config for deterministic discovery output (was previously set to 1.0)
54+
- Set top_p to 0.0 in all repo config files to force use of temperature setting by default.
55+
56+
- **Removed page image limit entirely across all IDP services**
57+
- removed image limits from multimodal inference steps (classification, extraction, assessment) following Amazon Bedrock API removal of image count restrictions. The system now processes all document pages without artificial truncation, with info logging to track image counts for monitoring purposes.
58+
- Resolves #147
59+
60+
- **Knowledge Base Vector Store Default Changed to S3 Vectors**
61+
- Changed default `KnowledgeBaseVectorStore` from `OPENSEARCH_SERVERLESS` to `S3_VECTORS` for cost-optimized deployments
62+
- S3 Vectors provides 40-60% lower storage costs with sub-second latency suitable for most use cases
63+
- OpenSearch Serverless remains available for applications requiring sub-millisecond query performance
64+
- No action required for existing deployments - only affects new stack deployments
65+
66+
### Fixed
67+
68+
- **UI: Document Schema Editor Regex Fields Not Persisting** - Fixed issue where Document Name Regex and Page Content Regex fields were not being saved in configuration or restored after page refresh. Fixes #151
69+
- **Document Schema Builder Enum Support** - Fixed enum value handling in schema builder to properly support enumeration constraints for attribute definitions
70+
- **Agentic Extraction Parameter Passing** - Fixed temperature and top_p parameters now correctly passed to agentic extraction service, enabling proper model behavior control
71+
- **Document Schema Builder UI Labels** - Enhanced field labels and formats in document schema builder for improved clarity and user experience
72+
- **Retry Mechanism Improvements** - Enhanced retry logic for more reliable error handling and recovery across document processing workflows
73+
- **Type Safety Enhancements** - Improved type annotations and fixed undefined items handling to prevent runtime errors
74+
75+
### Templates
76+
- us-west-2: `https://s3.us-west-2.amazonaws.com/aws-ml-blog-us-west-2/artifacts/genai-idp/idp-main_0.4.6.yaml`
77+
- us-east-1: `https://s3.us-east-1.amazonaws.com/aws-ml-blog-us-east-1/artifacts/genai-idp/idp-main_0.4.6.yaml`
78+
- eu-central-1: `https://s3.eu-central-1.amazonaws.com/aws-ml-blog-eu-central-1/artifacts/genai-idp/idp-main_0.4.6.yaml`
79+
80+
81+
## [0.4.5]
82+
83+
### Added
84+
85+
- **Document Split Classification Metrics for Evaluating Page-Level Classification and Document Segmentation**
86+
- Added `DocSplitClassificationMetrics` class for comprehensive evaluation of document splitting and classification accuracy
87+
- **Three Accuracy Types**: Page-level classification accuracy, split accuracy without order consideration, and split accuracy with exact page order matching
88+
- **Visual Reporting**: Generates markdown reports with color-coded indicators (🟢 Excellent, 🟡 Good, 🟠 Fair, 🔴 Poor), progress bars, and detailed section analysis tables
89+
- **Automatic Integration**: Integrates with evaluation service when ground truth and predicted sections are available
90+
- **Documentation**: Guide in `lib/idp_common_pkg/idp_common/evaluation/README.md` with usage examples, metric explanations, and best practices
91+
92+
- **Caching improvements to Agentic Extraction Service**
93+
- Optimized prompt caching by caching document context (text/images) on first LLM call, reducing token costs and quota consumption
94+
95+
- **Enhanced Bedrock Retry Logic for Agentic Extraction**
96+
- New `bedrock_utils.py` module with exponential backoff and comprehensive error handling
97+
- Improves agentic extraction reliability for transient failures and rate limiting
98+
99+
- **Review Agent Model Configuration**
100+
- Added `review_agent_model` parameter to enable separate model for reviewing extraction work
101+
- Defaults to main extraction model if not specified
102+
- Configurable through Web UI extraction settings
103+
104+
10105
### Fixed
11106

107+
- **Evaluation Output URI Fields Lost Across All Patterns - causing (a) missing Page Text Confidence content in UI, (2) failed Assessment step when reprocessing document after editing classes (No module named 'fitz')**
108+
- Fixed bug where `text_confidence_uri` was being set to null in evaluation output for all three patterns
109+
- Root cause: AppSync service `_appsync_to_document()` method incorrectly mapped page URIs, and evaluation functions overwrote correct documents with corrupted AppSync responses
110+
111+
- **UI: Metering Data Not Displayed During Document Processing**
112+
- Fixed UI subscription query missing `Metering` field, preventing real-time cost display
113+
- Users can now see estimated costs accumulate in real-time without manual page refresh
114+
115+
- **UI: Estimated Cost Panel Arrow Misalignment**
116+
- Fixed expand/contract arrow displaying above "Estimated Cost" heading
117+
118+
- **Agentic Extraction Reliability Improvements**
119+
- Updated Pydantic model serialization to use `model_dump(mode="json")` for proper JSON handling
120+
- Resolved linting issues and improved code quality across extraction modules
121+
122+
### Templates
123+
- us-west-2: `https://s3.us-west-2.amazonaws.com/aws-ml-blog-us-west-2/artifacts/genai-idp/idp-main_0.4.5.yaml`
124+
- us-east-1: `https://s3.us-east-1.amazonaws.com/aws-ml-blog-us-east-1/artifacts/genai-idp/idp-main_0.4.5.yaml`
125+
- eu-central-1: `https://s3.eu-central-1.amazonaws.com/aws-ml-blog-eu-central-1/artifacts/genai-idp/idp-main_0.4.5.yaml`
126+
127+
128+
## [0.4.4]
129+
130+
### Added
131+
132+
- **IDP CLI --from-code Flag for Local Development Deployment**
133+
- Added `--from-code` flag to `idp-cli deploy` command enabling deployment directly from local source code
134+
- Automatically builds project using `publish.py` script with streaming output for real-time build progress
135+
- **IDP CLI --no-rollback Flag for Stack Deployment Troubleshooting**
136+
- Added `--no-rollback` flag to `idp-cli deploy` command to disable automatic rollback on CloudFormation stack creation failure
137+
- When enabled, failed stacks remain in `CREATE_FAILED` state instead of rolling back, allowing inspection of failed resources for troubleshooting
138+
139+
- **Add support for prompt caching for Claude Haiku 4.5**
140+
141+
- **Add support for prompt caching for for EU region models**
142+
143+
### Fixed
144+
145+
- **Analytics Agent Schema Provider - Fixed Nested Attribute Column Display**
146+
- Fixed `schema_provider.py` to correctly display leaf-level nested columns instead of showing group-level attributes
147+
148+
- **IDP Agent Companion Chat UX improvements**
149+
- Improved speed of rendering chat response by buffering the agent tool responses.
150+
- Displaying agent tool queries and results in a modal with formatted results.
151+
152+
153+
### Templates
154+
- us-west-2: `https://s3.us-west-2.amazonaws.com/aws-ml-blog-us-west-2/artifacts/genai-idp/idp-main_0.4.4.yaml`
155+
- us-east-1: `https://s3.us-east-1.amazonaws.com/aws-ml-blog-us-east-1/artifacts/genai-idp/idp-main_0.4.4.yaml`
156+
- eu-central-1: `https://s3.eu-central-1.amazonaws.com/aws-ml-blog-eu-central-1/artifacts/genai-idp/idp-main_0.4.4.yaml`
157+
12158
## [0.4.3]
13159

14160
### Fixed

‎Makefile‎

Lines changed: 14 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -111,23 +111,32 @@ typecheck-pr:
111111

112112

113113
ui-lint:
114-
@echo "Checking UI lint"
115-
cd src/ui && npm ci --prefer-offline --no-audit && npm run lint
114+
@echo "Checking if UI lint is needed..."
115+
@CURRENT_HASH=$$(python3 -c "from publish import IDPPublisher; p = IDPPublisher(); print(p.get_directory_checksum('src/ui'))"); \
116+
STORED_HASH=$$(test -f src/ui/.checksum && cat src/ui/.checksum || echo ""); \
117+
if [ "$$CURRENT_HASH" != "$$STORED_HASH" ]; then \
118+
echo "UI code checksum changed - running lint..."; \
119+
cd src/ui && npm ci --prefer-offline --no-audit && npm run lint && \
120+
echo "$$CURRENT_HASH" > .checksum; \
121+
echo -e "$(GREEN)✅ UI lint completed and checksum updated$(NC)"; \
122+
else \
123+
echo -e "$(GREEN)✅ UI code checksum unchanged - skipping lint$(NC)"; \
124+
fi
116125

117126
ui-build:
118127
@echo "Checking UI build"
119128
cd src/ui && npm ci --prefer-offline --no-audit && npm run build
120129

121130
commit: lint test
122131
$(info Generating commit message...)
123-
export COMMIT_MESSAGE="$(shell q chat --no-interactive --trust-all-tools "Understand pending local git change and changes to be committed, then infer a commit message. Return this commit message only" | tail -n 1 | sed 's/\x1b\[[0-9;]*m//g')" && \
132+
export COMMIT_MESSAGE="$(shell kiro-cli chat --no-interactive --trust-all-tools "Understand pending local git change and changes to be committed, then infer a commit message. Return this commit message only on a single line." | grep ">" | tail -n 1 | sed 's/\x1b\[[0-9;]*m//g')" && \
124133
git add . && \
125134
git commit -am "$${COMMIT_MESSAGE}" && \
126135
git push
127136

128137
fastcommit: fastlint
129138
$(info Generating commit message...)
130-
export COMMIT_MESSAGE="$(shell q chat --no-interactive --trust-all-tools "Understand pending local git change and changes to be committed, then infer a commit message. Return this commit message only" | tail -n 1 | sed 's/\x1b\[[0-9;]*m//g')" && \
139+
export COMMIT_MESSAGE="$(shell kiro-cli chat --no-interactive --trust-all-tools "Understand pending local git change and changes to be committed, then infer a commit message. Return this commit message only on a single line." | grep ">" | tail -n 1 | sed 's/\x1b\[[0-9;]*m//g')" && \
131140
git add . && \
132141
git commit -am "$${COMMIT_MESSAGE}" && \
133-
git push
142+
git push

‎README.md‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,8 @@ White-glove customization, deployment, and integration support for production us
3434

3535
**Prefer AWS CDK?** This solution is also available as [GenAI IDP Accelerator for AWS CDK](https://github.com/cdklabs/genai-idp), providing the same functional capabilities through AWS CDK constructs for customers who prefer Infrastructure-as-Code with CDK.
3636

37+
**Prefer Terraform?** This solution is also available as [GenAI IDP Terraform](https://github.com/awslabs/genai-idp-terraform), providing the same functional capabilities as a Terraform module that integrates with existing infrastructure and supports customization through module variables.
38+
3739
## Key Features
3840

3941
- **Serverless Architecture**: Built entirely on AWS serverless technologies including Lambda, Step Functions, SQS, and DynamoDB
@@ -53,6 +55,7 @@ White-glove customization, deployment, and integration support for production us
5355
- **Extraction Confidence Assessment**: LLM-powered assessment of extraction confidence with multimodal document analysis
5456
- **Document Knowledge Base Query**: Ask questions about your processed documents
5557
- **IDP Accelerator Help Chat Bot**: Ask questions about the IDP code base or features
58+
- **MCP Integration**: Model Context Protocol integration enabling external applications like Amazon Quick Suite to access IDP data and analytics through AWS Bedrock AgentCore Gateway
5659

5760
## Architecture Overview
5861

@@ -121,7 +124,7 @@ idp-cli download-results \
121124
--output-dir ./results/
122125
```
123126

124-
**See [IDP CLI Documentation](./idp_cli/README.md)** for:
127+
**See [IDP CLI Documentation](./docs/idp-cli.md)** for:
125128
- CLI-based stack deployment and updates
126129
- Batch document processing
127130
- Complete evaluation workflows with baselines
@@ -162,7 +165,7 @@ For detailed deployment and testing instructions, see the [Deployment Guide](./d
162165

163166
- [Architecture](./docs/architecture.md) - Detailed component architecture and data flow
164167
- [Deployment](./docs/deployment.md) - Build, publish, deploy, and test instructions
165-
- [IDP CLI](./idp_cli/README.md) - Command line interface for batch processing and evaluation workflows
168+
- [IDP CLI](./docs/idp-cli.md) - Command line interface for batch processing and evaluation workflows
166169
- [Web UI](./docs/web-ui.md) - Web interface features and usage
167170
- [Agent Analysis](./docs/agent-analysis.md) - Natural language analytics and data visualization feature
168171
- [Custom MCP Agent](./docs/custom-MCP-agent.md) - Integrating external MCP servers for custom tools and capabilities
@@ -177,6 +180,7 @@ For detailed deployment and testing instructions, see the [Deployment Guide](./d
177180
- [Knowledge Base](./docs/knowledge-base.md) - Document knowledge base query feature
178181
- [Monitoring](./docs/monitoring.md) - Monitoring and logging capabilities
179182
- [IDP Accelerator Help Chat Bot](./docs/code-intelligence.md) - Chat bot for asking question about the IDP code base and features
183+
- [MCP Integration](./docs/mcp-integration.md) - Model Context Protocol integration for external applications like Amazon Quick Suite
180184
- [Reporting Database](./docs/reporting-database.md) - Analytics database for evaluation metrics and metering data
181185
- [Troubleshooting](./docs/troubleshooting.md) - Troubleshooting and performance guides
182186

‎VERSION‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
0.4.3
1+
0.4.6

0 commit comments

Comments
 (0)