feat: Add comprehensive CloudWatch dashboard for monitoring

Implements a complete CloudWatch dashboard with custom Lambda-powered widgets for real-time
monitoring and visualization of usage metrics. The dashboard provides comprehensive insights
into token consumption, user activity, code effectiveness, and cost tracking.

Key Features:
- DynamoDB single-table design for efficient metrics storage
- 10+ custom dashboard widgets with responsive layouts
- Real-time token usage and cost tracking
- User activity monitoring and analytics
- Code acceptance rate tracking
- Cache efficiency metrics
- Operations breakdown by type
- Lambda layer architecture for shared utilities

Technical Implementation:
- EventBridge scheduled metrics aggregation every 5 minutes
- Lambda functions for custom widget rendering
- DynamoDB TTL for automatic 30-day data retention
- Support for 10,000+ users with sub-second response times

Author: schuettc

.gitignore

@@ -49,4 +49,8 @@ parameters.json
 source/.ccwb-config/
 
 guidance-submission/
-tests/
+tests/
+assets/docs/planning
+
+.pytest_cache/
+.ruff_cache/

README.md

@@ -87,7 +87,7 @@ This guidance implements an Enterprise Authentication Pattern that enables organ
 - **Temporary AWS Credentials**: Eliminates long-lived credentials by providing session-based access to Amazon Bedrock
 - **Centralized Access Control**: Manage Claude Code access through your existing identity provider groups and policies
 - **Comprehensive Audit Logging**: Full CloudTrail integration for compliance and security monitoring
-- **Optional Usage Monitoring**: CloudWatch dashboards and metrics for tracking Claude Code usage across your organization
+- **Optional Usage Monitoring**: CloudWatch dashboards with metrics for tracking Claude Code usage, costs, and performance across your organization
 
 Step-by-step flow for end-users:
 
@@ -172,6 +172,7 @@ The guidance can be deployed in any AWS region that supports:
 ### Cross-Region Inference
 
 Claude Code uses Amazon Bedrock's cross-region inference for optimal performance and availability. During setup, you can:
+
 - Select your preferred Claude model (Opus, Sonnet, Haiku)
 - Choose a cross-region profile (US, Europe, APAC) for optimal regional routing
 - Select a specific source region within your profile for model inference
@@ -220,10 +221,12 @@ This creates the following AWS resources:
 - ECS Fargate cluster running OpenTelemetry collector
 - Application Load Balancer for OTLP ingestion
 - CloudWatch Log Groups and Metrics
-- CloudWatch Dashboard with usage analytics
-- Kinesis Data Firehose for streaming metrics to S3
-- Amazon Athena for SQL analytics on collected metrics
-- S3 bucket for long-term metrics storage
+- CloudWatch Dashboard with comprehensive usage analytics
+- DynamoDB table for metrics aggregation and storage
+- Lambda functions for custom dashboard widgets
+- Kinesis Data Firehose for streaming metrics to S3 (if analytics enabled)
+- Amazon Athena for SQL analytics on collected metrics (if analytics enabled)
+- S3 bucket for long-term metrics storage (if analytics enabled)
 
 ### Step 3: Create Distribution Package
 
@@ -326,10 +329,11 @@ Claude Code will automatically use your organization's authentication to access
 
 The optional CloudWatch dashboard provides:
 
-- **Usage Metrics**: Requests per user, model, and region
-- **Token Consumption**: Input/output tokens with cost estimation
-- **Performance Metrics**: Response times and error rates
-- **User Activity**: Active users and authentication patterns
+- **Token Consumption**: Input, output, and cache tokens tracked by user, model, and type
+- **Code Activity**: Lines of code written vs accepted, showing Claude Code effectiveness
+- **User Activity**: Active users, top consumers, and usage patterns throughout the day
+- **Cache Performance**: Cache hit rates and token savings from prompt caching
+- **Operations Breakdown**: Distribution of Claude Code operations (file edits, searches, reads, etc.)
 
 ## Troubleshooting
 
@@ -374,13 +378,6 @@ The guidance includes a comprehensive CLI tool (`ccwb`) for deployment and manag
 - [Monitoring and Telemetry Guide](/assets/docs/MONITORING.md) - Guide to deploying and using Claude Code Telemetry with OpenTelemetry
 - [Analytics Guide](/assets/docs/ANALYTICS.md) - Advanced analytics with Kinesis Firehose, S3 data lake, and Athena SQL queries
 
-**OIDC Provider Setup Guides:**
-
-- [Okta Setup](/assets/docs/providers/okta-setup.md)
-- [Microsoft Entra ID (Azure AD) Setup](/assets/docs/providers/microsoft-entra-id-setup.md)
-- [Auth0 Setup](/assets/docs/providers/auth0-setup.md)
-- [Cognito User Pool Setup](/assets/docs/providers/cognito-user-pool-setup.md)
-
 ### Supported OIDC Providers
 
 Detailed setup guides are available for:
@@ -408,18 +405,21 @@ poetry run pre-commit install
 #### How It Works
 
 When you commit changes, the following validations run automatically:
+
 - **YAML validation** checks syntax in all `.yaml` files
 - **CloudFormation validation** checks template structure and properties
 - **AWS CLI validation** validates templates against AWS specifications (if credentials are configured)
 
 The validation automatically catches:
+
 - YAML syntax errors that would cause deployment failures
 - CloudFormation template structure problems
 - Missing required parameters or invalid resource configurations
 
 #### Manual Validation (Optional)
 
 To run validation without committing:
+
 ```bash
 cd source
 poetry run pre-commit run --all-files

assets/docs/CLI_REFERENCE.md

@@ -30,8 +30,8 @@ The Claude Code with Bedrock CLI (`ccwb`) provides commands for IT administrator
 
 ```bash
 # Clone the repository
-git clone <repository-url>
-cd claude-code-auth-setup/source
+git clone [<repository-url>](https://github.com/aws-solutions-library-samples/guidance-for-claude-code-with-amazon-bedrock.git)
+cd guidance-for-claude-code-with-amazon-bedrock/source
 
 # Install dependencies
 poetry install

assets/docs/MONITORING.md

@@ -8,8 +8,6 @@ When you enable monitoring during deployment, the system creates infrastructure
 
 The monitoring system consists of several components working together. Claude Code sends metrics using the OpenTelemetry Protocol (OTLP) to an Application Load Balancer. The ALB forwards these metrics to an OTEL Collector running on ECS Fargate. The collector then converts the metrics to CloudWatch's Embedded Metric Format (EMF) and sends them to CloudWatch Metrics and Logs. Finally, a CloudWatch Dashboard visualizes these metrics.
 
-The OTEL Collector serves as a critical bridge because CloudWatch doesn't support OTLP metrics directly. Claude Code uses the industry-standard OTLP format, while CloudWatch requires its proprietary EMF format. The collector handles this translation, ensuring metrics flow smoothly from Claude Code to your CloudWatch dashboards.
-
 ## Implementation Details
 
 The monitoring infrastructure deploys several AWS resources to handle metric collection and visualization.
@@ -18,7 +16,7 @@ The core component runs as an ECS Fargate service using the AWS Distro for OpenT
 
 An Application Load Balancer sits in front of the ECS service, receiving OTLP metrics on port 4318. The ALB supports both HTTP and HTTPS protocols. When you provide a custom domain name during deployment, the system automatically creates an ACM certificate and configures HTTPS. Health checks monitor the collector's availability through the root endpoint.
 
-The CloudWatch Dashboard provides comprehensive visualization of your Claude Code usage. It displays metrics by user, tracks token consumption for both input and output, shows request counts broken down by model, monitors error rates and latency, and includes cost estimation widgets to help track spending.
+The CloudWatch Dashboard provides comprehensive visualization of your Claude Code usage. The dashboard uses Lambda functions and DynamoDB for efficient metrics collection and display, presenting real-time and historical usage data through custom widgets.
 
 ### Configuration
 
@@ -56,14 +54,6 @@ The deployment creates the complete monitoring infrastructure: a VPC with public
 
 If you provide a custom domain name and hosted zone ID during setup, the system automatically provisions an ACM certificate and configures HTTPS. This ensures encrypted transmission of metrics from Claude Code to your collector.
 
-After deployment completes, the package command handles all the configuration needed for end users:
-
-```bash
-poetry run ccwb package
-```
-
-This creates a distribution folder containing the credential process executable, configuration files, installation script, Claude Code telemetry settings, and platform-specific OTEL helper binaries that extract user attributes from authentication tokens.
-
 ## Claude Code Configuration
 
 The package command generates a `.claude/settings.json` file that configures Claude Code for telemetry collection. This file gets installed to the user's home directory and contains all the settings needed for monitoring.
@@ -105,13 +95,8 @@ Organizational dimensions provide additional context. The `department` field gro
 
 The CloudWatch Dashboard named `ClaudeCodeMonitoring` provides comprehensive visualization of your Claude Code metrics.
 
-The dashboard displays token usage metrics in multiple formats. Total token usage appears as time series graphs showing 5-minute and hourly intervals. Token breakdown by type (input, output, cache creation, cache read) helps you understand usage patterns. The dashboard includes pie charts showing token distribution over 30-day periods and stacked time series for analyzing trends.
-
-Cost tracking features estimate expenses based on token consumption. The dashboard calculates hourly costs using the configurable token price parameter (default $15 per million tokens). Single value widgets show today's cost, this week's cost, and this month's total cost. Time series graphs track cost trends with budget threshold annotations.
-
-Session and activity metrics track active sessions per hour, helping you understand usage patterns throughout the day. The dashboard also monitors Bedrock API calls broken down by model (Sonnet, Haiku, Opus), showing which models your users prefer.
-
-For detailed user analytics, the dashboard includes a section that directs you to AWS Athena. While the dashboard shows aggregate metrics, Athena queries enable user-specific analysis including top users by token consumption, usage by department or team, and cost allocation by user. The system stores pre-built queries in SSM Parameter Store for common analytics tasks.
+![Claude Code Monitoring Dashboard](/assets/images/ClaudeCodeDashboard.png)
+_Full dashboard view showing all metrics_
 
 ## End User Experience
 
@@ -131,17 +116,6 @@ CloudTrail tracking captures every Bedrock model invocation, storing detailed lo
 
 The monitoring dashboard includes several cost-related features. A separate Bedrock cost dashboard tracks AWS Billing charges for the Bedrock service. The main dashboard estimates costs based on token usage with configurable pricing (default $15 per million tokens). Real-time cost widgets show today's cost, this week's cost, and this month's total spending.
 
-The system includes pre-configured CloudWatch alarms for cost control:
-
-- **High Token Usage**: Alerts when usage exceeds 10 million tokens per hour
-- **Daily Token Threshold**: Triggers when daily usage exceeds approximately $1000 worth of tokens (66.7 million tokens at default pricing)
-- **High Bedrock Usage**: Alerts when Bedrock API calls exceed 1000 per hour
-- **Unusual Model Usage**: Monitors expensive model usage, alerting when it exceeds 50 million tokens per hour
-
-Bedrock tracking is enabled by default in the authentication stack. To disable it during deployment, set the CloudFormation parameter `EnableBedrockTracking` to false.
-
-The monitoring system tracks costs through multiple mechanisms. Token-based estimation provides immediate feedback on usage costs. The Bedrock cost dashboard shows actual AWS Billing charges for the Bedrock service. CloudWatch alarms help prevent unexpected spending by alerting on unusual usage patterns.
-
 ### Data Privacy
 
 The system collects only usage metrics, never capturing or transmitting conversation content between users and Claude. User attribution works through the email address from the OIDC token, providing clear accountability without excessive data collection. CloudWatch retains metrics for 15 months by default, though you can adjust this based on your retention policies. Beyond the email address used for attribution, no personally identifiable information is transmitted or stored.

assets/docs/providers/auth0-setup.md

@@ -3,6 +3,7 @@
 This guide walks you through setting up Auth0 from scratch to work with the AWS Cognito Identity Pool for Bedrock access.
 
 ## Table of Contents
+
 1. [Create Auth0 Account](#1-create-auth0-account)
 2. [Access Dashboard](#2-access-dashboard)
 3. [Create Native Application](#3-create-native-application)
@@ -60,6 +61,7 @@ If you don't have an Auth0 account:
 ### Step 4.1: Note the Client ID
 
 After creation, you'll see:
+
 - **Client ID**: Something like `aBcDeFgHiJkLmNoPqRsTuVwXyZ123456`
 - **Domain**: Your tenant domain like `your-name.auth0.com`
 
@@ -106,6 +108,7 @@ Click **Save Changes** at the bottom of the page
 ### Step 5.2: Create a Test User
 
 Fill in the form:
+
 - **Email**: `[email protected]`
 - **Password**: Enter a secure password
 - **Repeat Password**: Confirm the password
@@ -116,6 +119,7 @@ Click **Create**
 ### Step 5.3: Create Additional Users (Optional)
 
 Repeat to create more test users:
+
 - `[email protected]`
 - `[email protected]`
 
@@ -136,6 +140,7 @@ By default, all users have access to all applications in Auth0. To restrict acce
 ### Step 6.2: Enable Organizations (Optional)
 
 For enterprise deployments:
+
 1. Go to **Organizations**
 2. Create an organization
 3. Add users to the organization
@@ -147,10 +152,10 @@ For enterprise deployments:
 
 You now have everything needed for deployment:
 
-| Parameter | Your Value | Example |
-|-----------|------------|---------|
-| **Auth0Domain** | Your Auth0 domain | `your-name.auth0.com` |
-| **Auth0ClientId** | Your Client ID | `aBcDeFgHiJkLmNoPqRsTuVwXyZ123456` |
+| Parameter         | Your Value        | Example                            |
+| ----------------- | ----------------- | ---------------------------------- |
+| **Auth0Domain**   | Your Auth0 domain | `your-name.auth0.com`              |
+| **Auth0ClientId** | Your Client ID    | `aBcDeFgHiJkLmNoPqRsTuVwXyZ123456` |
 
 ### Use the values with ccwb init
 
@@ -202,20 +207,24 @@ Should return a JSON response with OIDC endpoints.
 ## Troubleshooting
 
 ### "Invalid redirect URI" Error
+
 - Ensure the callback URL is exactly: `http://localhost:8400/callback`
 - No trailing slashes or HTTPS
 
 ### "Unauthorized" Error
+
 - Check if user exists and password is correct
 - Verify application is active
 - Check for any Rules or Actions blocking access
 
 ### Can't Find Client ID
+
 1. Go to **Applications** → **Applications**
 2. Click on your application
 3. Client ID is at the top of the Settings tab
 
 ### Token Issues
+
 - Ensure Authorization Code grant type is enabled
 - Check that PKCE is not explicitly disabled
 - Verify refresh token settings
@@ -228,7 +237,7 @@ Once you've completed this Auth0 setup:
 
 1. Clone the repository:
    ```bash
-   git clone https://gitlab.aws.dev/schuettc/claude-code-setup.git
+   git clone https://github.com/aws-solutions-library-samples/guidance-for-claude-code-with-amazon-bedrock.git
    cd claude-code-setup
    poetry install
    ```
@@ -242,12 +251,14 @@ Once you've completed this Auth0 setup:
 ## Security Best Practices
 
 1. **Production Considerations**:
+
    - Enable MFA for all users
    - Use Auth0 Organizations for enterprise deployments
    - Set appropriate session and token lifetimes
    - Monitor logs regularly
 
 2. **Token Settings**:
+
    - Enable refresh token rotation
    - Set token expiration to 8 hours or less
    - PKCE is automatically enabled for native apps
@@ -265,6 +276,7 @@ Once you've completed this Auth0 setup:
 ### Custom Domain
 
 For production environments:
+
 1. Go to **Settings** → **Custom Domains**
 2. Add your domain (e.g., `auth.company.com`)
 3. Verify DNS settings
@@ -273,19 +285,24 @@ For production environments:
 ### Add Custom Claims
 
 To include user metadata in tokens:
+
 1. Go to **Actions** → **Flows** → **Login**
 2. Create a custom Action
 3. Add claims to the ID token:
    ```javascript
    exports.onExecutePostLogin = async (event, api) => {
      api.idToken.setCustomClaim('email', event.user.email);
-     api.idToken.setCustomClaim('department', event.user.user_metadata.department);
+     api.idToken.setCustomClaim(
+       'department',
+       event.user.user_metadata.department,
+     );
    };
    ```
 
 ### Enable Enterprise Connections
 
 For SSO with corporate identity providers:
+
 1. Go to **Authentication** → **Enterprise**
 2. Choose your connection type (SAML, OIDC, etc.)
 3. Configure according to your IdP requirements
@@ -300,4 +317,4 @@ For SSO with corporate identity providers:
 - Users: `https://manage.auth0.com/dashboard/users`
 - Logs: `https://manage.auth0.com/dashboard/logs`
 
-Remember to navigate to the correct tenant if you have multiple!
+Remember to navigate to the correct tenant if you have multiple!