Testing & Documentation

[jmgilman]

Testing & Documentation

Overview

Implement comprehensive testing strategy including unit tests, integration tests, and load tests, along with complete API documentation to ensure code quality, reliability, and maintainability.

Objective

Create a thorough testing suite that validates all components of the Certificate API Service, ensures performance requirements are met, and provides clear documentation for developers and operators.

Tasks

Unit Testing

API Handler Tests

• Create unit tests for certificate issuance handler in internal/api/handlers/certificates_test.go
• Test certificate renewal handler logic
• Test certificate status handler
• Test CA chain handler
• Mock dependencies (PCA client, database, auth)
• Test error scenarios:
  • Invalid input
  • Authentication failures
  • Authorization failures
  • PCA failures
  • Database failures

RBAC Permission Validation Tests

• Test permission checking logic in internal/auth/rbac_test.go
• Test domain glob pattern matching:
  • Exact matches
  • Wildcard patterns
  • Multiple SANs validation
• Test permission extraction from JWT claims
• Test authorization decisions for various scenarios

Certificate Profile Tests

• Test profile selection logic in internal/ca/profiles_test.go
• Validate client profile configuration
• Validate server profile configuration
• Validate machine profile configuration
• Test CA selection based on profile

Error Handling Tests

• Test error response formatting
• Test error code mapping
• Test error sanitization (no sensitive data leakage)
• Test HTTP status code mapping

Integration Testing

AWS PCA Integration Tests

• Set up test CA in AWS PCA (or use mock)
• Test certificate issuance flow end-to-end
• Test certificate renewal flow
• Test CA chain retrieval
• Test error handling for PCA failures
• Test retry logic with simulated failures

Database Integration Tests

• Test certificate storage and retrieval
• Test audit event logging
• Test transaction rollback scenarios
• Test concurrent access patterns
• Test connection pool behavior
• Use test database with migrations

JWT Validation Tests

• Test with valid Keycloak tokens (or mock JWKS)
• Test token expiration handling
• Test invalid signature handling
• Test JWKS caching behavior
• Test missing claims scenarios

End-to-End Certificate Issuance Tests

• Complete flow from request to certificate:

  func TestE2ECertificateIssuance(t *testing.T) {
    // 1. Create valid JWT token
    // 2. Make issuance request
    // 3. Verify certificate issued
    // 4. Check database record
    // 5. Verify audit log created
  }

Load Testing

Performance Benchmarks

• Create load test scenarios in test/load/scenarios.go
• Implement certificate issuance load test:
  • Target: 5 RPS sustained
  • Burst: 25 RPS for 30 seconds
• Implement certificate status lookup load test:
  • Target: high read throughput
• Use tools like vegeta or k6 for load testing

Performance Verification

• Verify certificate issuance <2 seconds (95th percentile)
• Verify certificate status lookup <500ms (95th percentile)
• Verify health check <100ms
• Monitor resource usage during load tests:
  • CPU utilization
  • Memory consumption
  • Database connections
  • Goroutine count

Stress Testing

• Test behavior under sustained load
• Test error handling under stress
• Test graceful degradation
• Test recovery after load spike

API Documentation

OpenAPI/Swagger Documentation

• Create OpenAPI specification in docs/api/openapi.yaml
• Document all endpoints:
  • POST /api/v1/certificates/issue
  • POST /api/v1/certificates/renew
  • GET /api/v1/certificates/{serial}
  • GET /api/v1/certificates/ca
  • GET /health
  • GET /ready
• Include request/response schemas
• Document authentication requirements
• Document error responses
• Add example requests and responses

API Documentation Content

• Document required headers (Authorization)
• Document request body schemas with validation rules
• Document response formats
• Document error codes and meanings (authoritative list in 07 Security & Validation):
  • invalid_token
  • insufficient_permissions
  • invalid_csr
  • domain_not_authorized
  • certificate_not_found
  • renewal_too_early
• Include curl examples for each endpoint

Developer Documentation

README Documentation

• Create comprehensive README.md:
  • Project overview
  • Architecture diagram
  • Quick start guide
  • Configuration options
  • Development setup
  • Testing instructions
  • Deployment guide

Code Documentation

• Add GoDoc comments to all exported functions
• Document package purposes
• Include usage examples in comments
• Document configuration structures

Operations Documentation

• Create runbook for common operations
• Document monitoring and alerting
• Include troubleshooting guide
• Document backup and recovery procedures
• Include performance tuning guidelines

Test Infrastructure

Test Fixtures and Helpers

• Create test certificate fixtures
• Create mock JWT tokens for testing
• Implement test database setup/teardown
• Create test helper functions:

  func NewTestServer(t *testing.T) *gin.Engine
  func NewTestDatabase(t *testing.T) *gorm.DB
  func GenerateTestJWT(t *testing.T, claims map[string]interface{}) string

CI/CD Integration

• Configure test execution in CI pipeline
• Set up code coverage reporting
• Configure linting (golangci-lint)
• Set up security scanning
• Ensure all tests run before merge

Acceptance Criteria

• Unit test coverage >80%
• All integration tests passing
• Load tests meet performance requirements
• API documentation complete and accurate
• README provides clear setup instructions
• All exported functions have GoDoc comments
• CI pipeline runs all tests successfully
• No race conditions detected

Technical Considerations

• Use Go testing package and testify for assertions
• Use mocks for external dependencies
• Use table-driven tests where appropriate
• Ensure tests are deterministic and repeatable
• Use test containers for database integration tests if needed
• Consider parallel test execution for speed
• Keep test execution time reasonable (<5 minutes for unit tests)

Dependencies

• Testing frameworks:
  • Standard library testing
  • github.com/stretchr/testify (assertions)
  • github.com/golang/mock or github.com/vektra/mockery (mocking)
• Load testing tools:
  • github.com/tsenart/vegeta or k6.io
• Documentation tools:
  • OpenAPI/Swagger tooling
  • godoc for code documentation

Testing Requirements

• Tests must be idempotent
• Tests must clean up after themselves
• Integration tests must use separate test database
• Load tests must not affect production systems
• Security tests must not expose vulnerabilities

Definition of Done

• All test categories implemented
• Code coverage meets target (>80%)
• Performance benchmarks documented
• API documentation reviewed and approved
• Developer documentation complete
• Operations runbook created
• All tests passing in CI
• Documentation published/accessible