Skip to content

Conversation

@ejc3
Copy link

@ejc3 ejc3 commented Nov 29, 2025

Note: This PR should be merged after #20 (request comparison feature)

Overview

This PR adds a comprehensive usage analytics dashboard inspired by Apple Screen Time, along with performance optimizations for displaying thousands of API requests.

Screenshot 2025-11-29 at 1 00 26 PM

New Features

📊 Usage Dashboard

A visual analytics interface showing Claude API usage patterns:

Weekly View

  • Bar chart showing token usage by day (Sunday-Saturday)
  • Color-coded by model (Purple=Opus, Blue=Sonnet, Green=Haiku)
  • Hover tooltips with per-model breakdown
  • Average usage line indicator
  • Displays total week usage

Today by Hour

  • Hourly activity distribution throughout the day
  • Current time indicator (vertical line at present hour)
  • Per-model breakdown in each hour
  • Identify peak usage times

Model Statistics

  • Horizontal bars showing proportional usage across models
  • Total tokens and request counts per model
  • Visual comparison of model distribution

Summary Metrics

  • Total tokens with K/M formatting
  • Request count
  • Average response time
  • Average tokens per request

⚡ Virtual Scrolling

  • Handles 5000+ requests without browser freezing
  • Only renders visible items + 10 overscan
  • Smooth 60fps scrolling
  • Uses TanStack Virtual v3.13.12

🗓️ Smart Date Navigation

  • Previous/Next day buttons
  • Shows "Today" when viewing current date
  • Shows date (e.g., "Nov 29") for other dates
  • Consistent labeling in selector and dashboard

🔧 New API Endpoints

  • GET /api/stats/hourly?start={UTC}&end={UTC} - Hourly breakdown
  • GET /api/stats/models?start={UTC}&end={UTC} - Model statistics
  • GET /api/requests/summary?start={UTC}&end={UTC} - Lightweight summaries
  • GET /api/requests/{id} - Single request details
  • GET /api/stats?start={UTC}&end={UTC} - Enhanced with per-model breakdowns

🌍 Timezone Handling

  • Browser calculates local day boundaries → UTC timestamps
  • Backend timezone-agnostic (receives exact UTC ranges)
  • SQLite uses datetime() for proper comparisons
  • "Today" always correct regardless of timezone

🗄️ Database Improvements

  • WAL mode for concurrent access
  • 5000ms busy timeout prevents lock errors
  • Per-model statistics in aggregates
  • Cache tokens included in calculations

Performance

Before: 5000 requests → 30s browser freeze
After: Instant render, 60fps scrolling

Stats loading: 3 parallel requests in ~200ms
Week navigation: 3x faster (only reloads when changing weeks)

Implementation

Frontend:

  • UsageDashboard.tsx - Dashboard component
  • UsageDashboard.css - Custom styling
  • Virtual scrolling with useVirtualizer
  • getLocalDayBoundaries() for timezone calc
  • Parallel stats loading

Backend:

  • GetHourlyStats() - Hourly breakdown with models
  • GetModelStats() - Model aggregates
  • Enhanced GetStats() with per-model data
  • All queries use datetime() for timezone awareness

Testing

✅ 5000+ requests
✅ Multiple timezones (PST/EST/UTC)
✅ Week navigation
✅ Model filtering
✅ Concurrent DB access

Breaking Changes

None - fully backward compatible

ejc3 added 16 commits December 20, 2025 11:45
- Replace sequential regex replacements with single-pass tokenizer in
  CodeViewer.tsx highlightCode() function
- The old approach applied patterns sequentially, causing later patterns
  to match numbers inside class attributes (e.g., "400" in "text-purple-400")
- New approach: build combined regex, iterate matches once, escape HTML
  on matched tokens only
- Also fix escapeHtml in formatters.ts to not use document.createElement
  (fails during SSR) and simplify formatLargeText to avoid over-formatting
- Add quote escaping to escapeHtml in CodeViewer.tsx for XSS protection
- Fix string regex patterns to properly handle escaped quotes
- Use template literal for paragraph wrapping in formatLargeText
- Add vitest and tests for escapeHtml, formatLargeText, and string patterns
- Change combined "X tokens" to separate "X in" / "Y out" display
- Makes it clearer how many tokens are uploaded vs generated
- Helps users understand conversation growth per turn
- Show total input tokens (cached + non-cached) instead of just non-cached
- Change cache display from absolute number to percentage
- "68,446 in 100% cached" instead of "1 in 153,525 cached"
Add guard to ensure denominator is non-zero before calculating the
cache percentage. This prevents NaN when both input_tokens and
cache_read_input_tokens are 0.
- Add Compare button in header to enter compare mode
- Allow selecting 2 requests via checkboxes for side-by-side comparison
- Create RequestCompareModal component with:
  - Summary stats (added/removed/modified/unchanged messages)
  - Side-by-side request metadata comparison
  - Message diff view with color-coded changes
  - System prompt comparison
  - Tools comparison (added/removed/common)
- Sticky compare mode banner that persists while scrolling
- Button label changes based on state (Compare / Exit Compare)
RequestCompareModal:
- Add text diff view with side-by-side line comparison (LCS algorithm)
- Show system prompt and tools in diff, not just messages
- Add size breakdown: system prompt, tools, messages in KB
- Show cache read/creation tokens separately
- Add message size (KB) to each message row in structured view
- Add download options: .diff, .json, .md formats
- Add "Side-by-Side" export for external diff tools
- Toggle between Structured and Text Diff views

_index.tsx:
- Fix cache display to only show when > 0
Remove hardcoded modelProviderMap and initializeModelProviderMap function.
Use simple prefix matching instead:
- claude* -> anthropic
- gpt*, o1*, o3* -> openai

This automatically handles new model versions without code changes.
Backend:
- Add /api/requests/summary endpoint returning lightweight RequestSummary
- RequestSummary includes only: id, timestamp, model, status, usage, responseTime
- Skip parsing heavy body/headers JSON for faster list loading

Frontend:
- Replace react-virtuoso with @tanstack/react-virtual for 60fps scrolling
- Load summaries first for fast initial render, preload full details in background
- Cache full request details in Map for instant row clicks
- Use window scrolling instead of inner container scroll
- Enable WAL mode for concurrent reads during writes
- Set busy timeout to 5 seconds instead of immediate failure
- Use NORMAL synchronous mode for better performance
Dashboard Features:
- Add UsageDashboard component with Apple Screen Time-inspired design
- Display daily token usage, request count, and average response time
- Weekly bar chart showing last 7 days with stacked model breakdown
- Hourly bar chart for selected day with 24-hour view
- Model usage breakdown with color-coded bars
  - Opus: purple gradient (#9333ea)
  - Sonnet: blue gradient (#3b82f6)
  - Haiku: green gradient (#10b981)
- Interactive tooltips showing per-model token counts
- Date navigation with previous/next day arrows
- Dynamic week labels (e.g., "Nov 21 - 27" or "THIS WEEK")
- Model filter tabs integrated into requests box header

Backend API Changes:
- Add GET /api/stats endpoint for dashboard statistics
- Add GET /api/requests/summary endpoint for lightweight request list
- Add GET /api/requests/{id} endpoint to fetch individual requests
- Add per-model token breakdown to DailyTokens and HourlyTokens
- Add ModelStats struct for tracking tokens/requests per model
- Track model breakdown in hourly and daily aggregation maps
- Switch from date-string filtering to UTC time-range filtering
- Accept start/end UTC timestamps instead of date parameters
- Remove default 100-request limit, fetch all requests for selected day

Timezone Handling:
- Browser calculates local day boundaries (12:00 AM to 11:59 PM)
- Convert local time boundaries to UTC before sending to backend
- Backend queries using exact UTC timestamp ranges
- Fix week label parsing to avoid timezone shifts
- Stats query fetches 7-day range (selected date - 6 days)
- Requests query fetches single day range
- Ensures "today" is intuitive to user's local timezone

Performance Optimizations:
- Fetch individual requests on-demand instead of loading all 10k+
- Cache fetched request details in client state
- Use lightweight summary endpoint for initial list view
- Only request list refreshes when filter changes, stats remain static
- Fast aggregation queries using maps for O(1) lookups

Routes:
- Add /api/stats Remix route proxying to Go backend
- Add /api/requests/{id} Remix route for single request fetch
- Add getWeekBoundaries() helper to calculate Sunday-Saturday week
- Track current week start to detect week changes
- Only reload stats when navigating to a different week
- Always reload requests for hourly chart on date change
- Show actual date in navigation (bold when today) instead of 'Today' label
- Weekly chart bars now stay stable while navigating within the same week
- Create /api/stats/hourly endpoint for single-day data
- Move hourly breakdown and model stats to hourly endpoint
- Keep /api/stats for weekly overview (daily aggregates only)
- Frontend loads both endpoints in parallel
- Add HourlyStatsResponse type for hourly endpoint response
- Model stats now only shown for selected date, not entire week
- Split loadStats into loadWeeklyStats and loadHourlyStats
- When navigating to new week: load both weekly and hourly stats
- When navigating within same week: only load hourly stats
- Prevents frontend lockup by not reloading weekly data unnecessarily
- Hourly chart now updates correctly when clicking previous/next
Backend changes:
- Add /api/stats/models endpoint for model-specific statistics
- Use datetime() function in SQLite queries for timezone-aware comparisons
- Fix queries to properly handle RFC3339 timestamps with timezone offsets
- Add GetModelStats and GetHourlyStats to storage interface

Frontend changes:
- Implement TanStack Virtual for requests list to handle thousands of items
- Add virtual scrolling with 600px container and 120px estimated item size
- Show "Today" vs specific date (e.g., "Nov 29") in date selector and stats
- Remove debug console.log statements
- Update UsageDashboard to show relative date labels

Performance improvements:
- Only render visible request items (10 overscan) instead of all 5000+
- Prevent browser freezing when navigating between dates
- Efficient scrolling with virtualized rendering
Frontend changes:
- Replace date string format with full UTC timestamp boundaries
- Calculate local day start/end in browser (e.g., Nov 29 PST = 08:00-08:00 UTC)
- Send start/end timestamps instead of date strings to API
- Backend now receives exact UTC time ranges for client's local day

Backend changes:
- Update GetHourlyStats to accept start/end timestamps instead of date
- Update GetModelStats to accept start/end timestamps instead of date
- Remove server-side date parsing and timezone interpretation
- Backend is now completely timezone-agnostic

This ensures "Today" shows correct data regardless of client timezone.
No more date/timezone confusion between client and server.
When the dashboard loads with no requests for today, it automatically
fetches the most recent date with data and navigates to that date.

- Add GetLatestRequestDate to storage interface and SQLite implementation
- Add /api/requests/latest-date endpoint
- Update frontend to check for empty data and snap to latest date
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant