docs: Update EventBridge integration to use native S3 events

[drernie]

Summary

Update EventBridge integration documentation to reflect AWS's native S3 event support in EventBridge, removing obsolete CloudTrail requirements and adding critical missing configuration steps.

Key Changes

1. Remove CloudTrail Requirements

• ✅ Removed CloudTrail as a prerequisite
• ✅ Updated event patterns to use native S3 events (Object Created, Object Deleted)
• ✅ Removed CloudTrail-specific troubleshooting and limitations
• ✅ Updated performance considerations (sub-second latency vs 1-5 seconds)

2. Fix Input Transformer Configuration

• ✅ Corrected input path to use native S3 event structure
• ✅ Simplified input template with essential fields only
• ✅ Added proper eventSource: "aws:s3" field

3. Add Critical Missing Steps

• ✅ Step 2: Enable EventBridge on S3 bucket (critical prerequisite)
• ✅ Step 7: Configure IAM/KMS permissions for encrypted SNS topics
• ✅ Step 11: Comprehensive end-to-end verification with CloudWatch metrics

4. Add Important Configuration Details

• ✅ Raw message delivery settings for different SQS queues
• ✅ KMS permissions for EventBridge to publish to encrypted SNS topics
• ✅ AWS CLI commands for verification and monitoring

Benefits

• Simpler setup: Fewer prerequisites and steps to follow
• Lower latency: Sub-second delay vs 1-5 seconds with CloudTrail
• Lower cost: No CloudTrail data events charges
• More reliable: Direct S3 events without CloudTrail processing layer
• Complete: All critical steps now documented

Structure Changes

Guide expanded from 8 steps to 11 steps:

1. Create SNS Topic
2. Enable EventBridge on S3 Bucket ⭐ NEW
3. Create EventBridge Rule
4. Configure Event Pattern
5. Configure Event Target
6. Set Up Input Transformer
7. Configure IAM Permissions ⭐ NEW
8. Save and Test the Rule
9. Configure Quilt
10. Initial Indexing
11. Comprehensive Verification ⭐ NEW

Testing

Documentation changes only. The configuration has been validated to work with:

• Native S3 EventBridge events (no CloudTrail required)
• Encrypted SNS topics with KMS
• Multiple SQS queue subscriptions with different raw message delivery settings

Commits

• 2360a174 - Remove CloudTrail requirements, update to native S3 events
• fe65f831 - Fix input transformer and add configuration details
• 4d112516 - Add critical S3 EventBridge setup steps and verification

---

🤖 Generated with Claude Code

Co-Authored-By: Claude noreply@anthropic.com

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 43.22%. Comparing base (2abb847) to head (4d11251).

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #4686   +/-   ##
=======================================
  Coverage   43.22%   43.22%           
=======================================
  Files         797      797           
  Lines       32003    32003           
  Branches     5699     5699           
=======================================
  Hits        13833    13833           
  Misses      16176    16176           
  Partials     1994     1994           

Flag	Coverage Δ
api-python	91.96% <ø> (ø)
catalog	19.43% <ø> (ø)
lambda	96.62% <ø> (ø)
py-shared	98.18% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[drernie]

Closing this PR to create a clean version without the SUMMARY.md change (moved to #4687)