Skip to content

Commit 1b13d03

Browse files
authored
Merge pull request #64 from JohanDevl/develop
feat: major feature release with OAuth 2.0 authentication and individual watch history export
2 parents 01934df + 4afdf2e commit 1b13d03

File tree

23 files changed

+3753
-129
lines changed

23 files changed

+3753
-129
lines changed

.gitignore

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -218,6 +218,38 @@ config/config.toml
218218
# But keep the example file
219219
!config/config.example.toml
220220

221+
# OAuth credential files
222+
config/credentials.enc
223+
config/credentials.*
224+
config/*credentials*
225+
config/*.key
226+
config/token.json
227+
config/*.token
228+
**/access_token*
229+
**/refresh_token*
230+
231+
# Security and encryption keys
232+
*.enc
233+
*.key
234+
*.pem
235+
*.p12
236+
*.pfx
237+
encryption_key*
238+
auth_key*
239+
240+
# Environment files with secrets
241+
.env
242+
.env.local
243+
.env.production
244+
.env.test
245+
*.env
246+
247+
# Docker secrets and credentials
248+
secrets/
249+
.secrets/
250+
docker-secrets/
251+
.docker-secrets/
252+
221253
# Go specific ignores
222254
export_trakt
223255
export-trakt

CLAUDE.md

Lines changed: 254 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,254 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
## Project Overview
6+
7+
Export Trakt 4 Letterboxd is a Go-based application that exports movie data from Trakt.tv to Letterboxd-compatible CSV files. The application features enterprise-grade architecture with performance optimization, comprehensive security, monitoring, and internationalization support.
8+
9+
## Core Development Commands
10+
11+
### Testing
12+
```bash
13+
# Run all tests
14+
go test ./...
15+
16+
# Run tests with coverage
17+
go test -coverprofile=coverage.out ./...
18+
go tool cover -html=coverage.out -o coverage.html
19+
20+
# Run coverage script (enforces 70% minimum coverage)
21+
./scripts/coverage.sh
22+
23+
# Run benchmarks
24+
go test -bench=. ./pkg/performance/
25+
```
26+
27+
### Building
28+
```bash
29+
# Development build
30+
go build -o export_trakt ./cmd/export_trakt/
31+
32+
# Production build with optimizations
33+
go build -ldflags "-w -s" -o export_trakt ./cmd/export_trakt/
34+
35+
# Cross-platform builds
36+
GOOS=linux GOARCH=amd64 go build -o export_trakt_linux ./cmd/export_trakt/
37+
GOOS=windows GOARCH=amd64 go build -o export_trakt_windows.exe ./cmd/export_trakt/
38+
GOOS=darwin GOARCH=amd64 go build -o export_trakt_darwin ./cmd/export_trakt/
39+
```
40+
41+
### Docker Development
42+
```bash
43+
# Build Docker image
44+
docker build -t export-trakt-dev .
45+
46+
# Run with Docker Compose profiles
47+
docker compose --profile dev --profile run-all up --build # Development build
48+
docker compose --profile run-all up # Production image
49+
docker compose --profile schedule-6h up -d # Production scheduler
50+
docker compose --profile dev --profile schedule-test up -d --build # Test scheduler
51+
```
52+
53+
### Application Usage
54+
```bash
55+
# OAuth Authentication (required for first use)
56+
./export_trakt auth
57+
58+
# Check token status
59+
./export_trakt token-status
60+
61+
# One-time export (default aggregated mode)
62+
./export_trakt --run --export all --mode complete
63+
64+
# Individual watch history export (all viewing events)
65+
./export_trakt --run --export watched --history-mode individual
66+
67+
# Aggregated export (one entry per movie, original behavior)
68+
./export_trakt --run --export watched --history-mode aggregated
69+
70+
# Scheduled export
71+
./export_trakt --schedule "0 */6 * * *" --export all --mode complete
72+
73+
# Validate configuration
74+
./export_trakt validate
75+
76+
# Security validation
77+
./export_trakt --validate-security
78+
```
79+
80+
## Architecture Overview
81+
82+
### Core Packages Structure
83+
- **cmd/export_trakt/** - Main application entry point with CLI handling
84+
- **pkg/api/** - Trakt.tv API client with optimized HTTP client and retry logic
85+
- **pkg/auth/** - OAuth 2.0 authentication and token management
86+
- **pkg/config/** - Configuration management with TOML support
87+
- **pkg/export/** - CSV export functionality for Letterboxd format
88+
- **pkg/scheduler/** - Cron-based scheduling system
89+
- **pkg/logger/** - Structured logging with i18n support
90+
- **pkg/security/** - Security features including encryption, audit logging, keyring management
91+
- **pkg/performance/** - Performance optimization features (worker pools, LRU cache, streaming)
92+
- **pkg/monitoring/** - Observability with Prometheus metrics and OpenTelemetry tracing
93+
- **pkg/retry/** - Resilient API calls with circuit breaker and exponential backoff
94+
- **pkg/i18n/** - Internationalization support (English, French, German, Spanish)
95+
96+
### Key Architectural Patterns
97+
98+
**Performance Layer**: The application uses a 3-tier performance optimization:
99+
1. **Worker Pool System** (`pkg/performance/pool/`) - Concurrent processing of API requests
100+
2. **LRU Cache** (`pkg/performance/cache/`) - Caches API responses to reduce calls by 70-90%
101+
3. **Streaming Processor** (`pkg/streaming/`) - Memory-efficient processing of large datasets
102+
103+
**Security Layer**: Enterprise-grade security with:
104+
- **OAuth 2.0 Authentication** (`pkg/auth/`) - Modern OAuth 2.0 authentication with automatic token refresh
105+
- **Keyring Integration** (`pkg/security/keyring/`) - Secure credential storage across multiple backends
106+
- **Encryption** (`pkg/security/encryption/`) - AES encryption for sensitive data
107+
- **Audit Logging** (`pkg/security/audit/`) - Comprehensive security event logging
108+
- **Rate Limiting** (`pkg/security/ratelimit.go`) - API rate limiting protection
109+
110+
**Resilience Layer**: Built-in resilience patterns:
111+
- **Circuit Breaker** (`pkg/retry/circuit/`) - Prevents cascading failures
112+
- **Exponential Backoff** (`pkg/retry/backoff/`) - Smart retry logic
113+
- **Checkpoints** (`pkg/resilience/checkpoints/`) - Resume interrupted operations
114+
115+
**Monitoring Layer**: Full observability stack:
116+
- **Prometheus Metrics** (`pkg/monitoring/metrics/`) - Application metrics
117+
- **OpenTelemetry Tracing** (`pkg/monitoring/tracing/`) - Distributed tracing
118+
- **Health Checks** (`pkg/monitoring/health/`) - Application health monitoring
119+
- **Alerting** (`pkg/monitoring/alerts/`) - Alert management
120+
121+
## Export Modes
122+
123+
The application supports two distinct export modes for watched movies:
124+
125+
### Aggregated Mode (Default)
126+
- **Behavior**: One entry per unique movie with the most recent watch date
127+
- **Use Case**: Standard export compatible with original Letterboxd import expectations
128+
- **Configuration**: `history_mode = "aggregated"` or `--history-mode aggregated`
129+
- **Output**: Single CSV row per movie, rewatch flag based on total play count
130+
131+
### Individual History Mode (New)
132+
- **Behavior**: One entry per viewing event with complete watch history
133+
- **Use Case**: Detailed tracking of all viewing dates and proper rewatch sequences
134+
- **Configuration**: `history_mode = "individual"` or `--history-mode individual`
135+
- **Output**: Multiple CSV rows per movie (one per viewing), chronological rewatch tracking
136+
- **Data Source**: Uses `/sync/history/movies` API endpoint with 'watch' and 'scrobble' actions
137+
- **Benefits**:
138+
- Complete viewing history preservation
139+
- Accurate rewatch tracking (first watch = false, subsequent = true)
140+
- Chronological sorting (most recent first)
141+
- Same rating applied consistently across all viewings of a movie
142+
143+
**Example Individual Mode Output:**
144+
```csv
145+
Title,Year,WatchedDate,Rating10,imdbID,tmdbID,Rewatch
146+
Cars,2006,2025-07-10,7,tt0317219,920,true
147+
Cars,2006,2024-12-01,7,tt0317219,920,false
148+
```
149+
150+
## Configuration Management
151+
152+
Configuration is handled through TOML files with the following priority:
153+
1. Command-line flags
154+
2. Environment variables
155+
3. `config/config.toml` file
156+
4. `config/config.example.toml` defaults
157+
158+
Key configuration files:
159+
- `config/config.example.toml` - Complete configuration template with security features
160+
- `config/performance.toml` - Performance optimization settings
161+
162+
## Security Considerations
163+
164+
The application implements multiple security layers:
165+
- **OAuth 2.0 Authentication**: Modern authentication flow with PKCE and automatic token refresh
166+
- **Credential Management**: Uses system keyring, environment variables, or encrypted files
167+
- **Multi-Backend Token Storage**: Supports system keyring, file encryption, and environment variables
168+
- **HTTPS Enforcement**: Configurable requirement for HTTPS-only communications
169+
- **Input Validation**: Comprehensive sanitization and validation
170+
- **Audit Logging**: Detailed security event logging with configurable retention
171+
- **File Permissions**: Secure file creation and access controls
172+
173+
## Testing Strategy
174+
175+
The codebase maintains high test coverage (78%+) with:
176+
- **Unit Tests**: All major packages have comprehensive test coverage
177+
- **Integration Tests**: End-to-end workflow testing
178+
- **Benchmark Tests**: Performance testing for critical paths
179+
- **Security Tests**: Security feature validation
180+
181+
Critical test files:
182+
- `pkg/performance/benchmarks_test.go` - Performance benchmarks
183+
- `pkg/security/manager_test.go` - Security feature tests
184+
- `pkg/api/trakt_test.go` - API client tests
185+
186+
## Performance Characteristics
187+
188+
Recent performance optimizations deliver:
189+
- **10x throughput improvement** via worker pool system
190+
- **70-90% API call reduction** through LRU caching
191+
- **80% memory reduction** with streaming processing
192+
- **Sub-second response times** for most operations
193+
194+
## Internationalization
195+
196+
The application supports multiple languages through the `pkg/i18n` package:
197+
- Message keys are used throughout the codebase
198+
- Translation files are located in `locales/` directory
199+
- Supported languages: English, French, German, Spanish
200+
201+
## Error Handling
202+
203+
Comprehensive error handling system:
204+
- **Error Types** (`pkg/errors/types/`) - Structured error definitions
205+
- **Error Handlers** (`pkg/errors/handlers/`) - Centralized error handling
206+
- **Error Recovery** (`pkg/errors/recovery/`) - Panic recovery mechanisms
207+
- **Validation** (`pkg/errors/validation/`) - Input validation with detailed error messages
208+
209+
## Development Workflow
210+
211+
1. **Make Changes**: Edit code following existing patterns
212+
2. **Run Tests**: Execute `go test ./...` to ensure no regressions
213+
3. **Check Coverage**: Use `./scripts/coverage.sh` to verify coverage meets 70% threshold
214+
4. **Build**: Use `go build -o export_trakt ./cmd/export_trakt/`
215+
5. **Test Integration**: Use Docker Compose profiles for integration testing
216+
6. **Security Check**: Run `./export_trakt --validate-security` for security validation
217+
218+
## CI/CD Pipeline
219+
220+
The project uses GitHub Actions with:
221+
- **Go Tests** (.github/workflows/go-tests.yml) - Testing with 57% minimum coverage
222+
- **Docker Build** (.github/workflows/docker-build.yml) - Multi-platform container builds
223+
- **Security Scan** (.github/workflows/security-scan.yml) - gosec, trivy, and dependency scanning
224+
- **Release** (.github/workflows/release.yml) - Automated release creation
225+
226+
## Common Patterns
227+
228+
**Structured Logging**: All log messages use structured format with message keys:
229+
```go
230+
log.Info("export.starting", map[string]interface{}{
231+
"export_type": exportType,
232+
"timestamp": time.Now(),
233+
})
234+
```
235+
236+
**Configuration Access**: Use dependency injection for configuration:
237+
```go
238+
func NewClient(cfg *config.Config, log logger.Logger) *Client {
239+
return &Client{config: cfg, logger: log}
240+
}
241+
```
242+
243+
**Error Handling**: Use structured error types with context:
244+
```go
245+
return fmt.Errorf("failed to export movies: %w", err)
246+
```
247+
248+
**Performance Optimization**: Leverage worker pools for concurrent operations:
249+
```go
250+
pool := workerPool.New(cfg.Performance.WorkerCount)
251+
defer pool.Close()
252+
```
253+
254+
When making changes, always follow existing architectural patterns and maintain the security-first approach. The codebase emphasizes reliability, performance, and security over simplicity.

README.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -220,6 +220,30 @@ Export_Trakt_4_Letterboxd/
220220
| `shows` | TV show data | ⚠️ Limited support |
221221
| `all` | Everything above | ✅ Complete migration |
222222

223+
### 🎯 Watch History Modes
224+
225+
The `watched` export type supports two distinct modes:
226+
227+
| Mode | Behavior | CLI Usage |
228+
| ------------- | ---------------------------------------- | ----------------------------------- |
229+
| `aggregated` | One entry per movie (original behavior) | `--history-mode aggregated` |
230+
| `individual` | One entry per viewing event | `--history-mode individual` |
231+
232+
**Individual Mode Benefits:**
233+
- 📅 Complete viewing history with all dates
234+
- 🔄 Accurate rewatch tracking (chronological order)
235+
- 📊 Multiple entries for rewatched movies
236+
- ⏰ Preserves exact viewing timestamps
237+
238+
**Example:**
239+
```bash
240+
# Get complete viewing history (all watch events)
241+
./export_trakt --run --export watched --history-mode individual
242+
243+
# Standard export (one entry per movie)
244+
./export_trakt --run --export watched --history-mode aggregated
245+
```
246+
223247
## 🌍 Internationalization
224248

225249
Supported languages:

0 commit comments

Comments
 (0)