diff --git a/.dockerignore b/.dockerignore new file mode 100644 index 000000000..826814cee --- /dev/null +++ b/.dockerignore @@ -0,0 +1,88 @@ +# .dockerignore for Hermes backend build +# Only include what's needed for the Go server binary + +# Exclude testing infrastructure +testing/ +tests/ + +# Exclude documentation +docs-internal/ +*.md +!README.md + +# Exclude build artifacts +build/ +/hermes +*.exe +*.dll +*.so +*.dylib +*.test +*.out +coverage*.out +coverage*.html + +# Exclude config files with secrets +config.hcl +credentials.json +token.json +.env* + +# Exclude version control +.git/ +.github/ +.gitignore +.gitattributes + +# Exclude test outputs and logs +workspace.test +test_results.log +test_output.log +test_final.log +*.log +*.patch + +# Exclude macOS and OS files +.DS_Store +Thumbs.db + +# Exclude Python scripts +*.py +*.sh +refactor_*.py +fix_*.py +scripts/ + +# Exclude web development files (keep only dist/ and web.go) +web/node_modules/ +web/.yarn/ +web/.pnp.* +web/.npm/ +web/.eslintcache +web/tests/ +web/mirage/ +web/tmp/ +web/.cache/ +web/babel.config.js +web/ember-cli-build.js +web/package.json +web/yarn.lock +web/tsconfig.json +web/testem.js +web/tailwind.config.js +web/.template-lintrc.js +web/.eslintrc.js +web/.prettierrc.js +web/types/ +web/vendor/ + +# Keep only web/dist (needed for go:embed) and web/web.go +!web/dist/ +!web/web.go + +# Exclude IDE +.idea/ +.vscode/ +*.swp +*.swo +*~ diff --git a/.env.example b/.env.example new file mode 100644 index 000000000..187babe8e --- /dev/null +++ b/.env.example @@ -0,0 +1,26 @@ +# Minimal .env example for local development and testing +# Copy to .env and fill in your actual values + +# Required for web frontend +HERMES_WEB_GOOGLE_OAUTH2_CLIENT_ID=your-client-id.apps.googleusercontent.com + +# Required for backend Google API access +GOOGLE_APPLICATION_CREDENTIALS=./credentials.json + +# Required for search functionality +HERMES_WEB_ALGOLIA_APP_ID=your-algolia-app-id +HERMES_WEB_ALGOLIA_SEARCH_API_KEY=your-search-api-key +ALGOLIA_ADMIN_API_KEY=your-admin-api-key + +# Optional: Override default index names +HERMES_WEB_ALGOLIA_DOCS_INDEX_NAME=hermes_docs +HERMES_WEB_ALGOLIA_DRAFTS_INDEX_NAME=hermes_drafts +HERMES_WEB_ALGOLIA_INTERNAL_INDEX_NAME=hermes_internal +HERMES_WEB_ALGOLIA_PROJECTS_INDEX_NAME=hermes_projects + +# Optional: Database (uses Docker defaults if not set) +DATABASE_URL=postgresql://postgres:postgres@localhost:5432/hermes?sslmode=disable + +# Optional: Application settings +HERMES_WEB_SHORT_LINK_BASE_URL=http://localhost:8000 +LOG_LEVEL=info diff --git a/.env.template b/.env.template new file mode 100644 index 000000000..c428498f1 --- /dev/null +++ b/.env.template @@ -0,0 +1,90 @@ +# Hermes Environment Variables Template +# Copy this file to .env and fill in your actual values +# WARNING: Never commit .env files with real credentials! + +# ============================================================================= +# Google Workspace Integration +# ============================================================================= + +# Google OAuth 2.0 Client ID (for web frontend authentication) +# Get this from Google Cloud Console > APIs & Services > Credentials +HERMES_WEB_GOOGLE_OAUTH2_CLIENT_ID=your-client-id.apps.googleusercontent.com + +# Google Service Account Credentials (for backend API access) +# Path to your service account JSON file (e.g., ./credentials.json) +# Download from Google Cloud Console > IAM & Admin > Service Accounts +GOOGLE_APPLICATION_CREDENTIALS=./credentials.json + +# ============================================================================= +# Algolia Search Configuration +# ============================================================================= + +# Algolia Application ID +HERMES_WEB_ALGOLIA_APP_ID=your-algolia-app-id + +# Algolia Search API Key (public, read-only key for frontend) +HERMES_WEB_ALGOLIA_SEARCH_API_KEY=your-algolia-search-api-key + +# Algolia Admin API Key (for backend indexing operations) +# Keep this secret! Only use in backend/testing +ALGOLIA_ADMIN_API_KEY=your-algolia-admin-api-key + +# Algolia Index Names (customize per environment: dev, staging, prod) +HERMES_WEB_ALGOLIA_DOCS_INDEX_NAME=hermes_docs_dev +HERMES_WEB_ALGOLIA_DRAFTS_INDEX_NAME=hermes_drafts_dev +HERMES_WEB_ALGOLIA_INTERNAL_INDEX_NAME=hermes_internal_dev +HERMES_WEB_ALGOLIA_PROJECTS_INDEX_NAME=hermes_projects_dev + +# ============================================================================= +# Application Configuration +# ============================================================================= + +# Short Link Base URL (for generating short URLs) +HERMES_WEB_SHORT_LINK_BASE_URL=http://localhost:8000 + +# Google Analytics Tag ID (optional, for tracking) +HERMES_WEB_GOOGLE_ANALYTICS_TAG_ID= + +# ============================================================================= +# Database Configuration +# ============================================================================= + +# PostgreSQL connection string +# Default for local Docker: postgresql://postgres:postgres@localhost:5432/hermes?sslmode=disable +DATABASE_URL=postgresql://postgres:postgres@localhost:5432/hermes?sslmode=disable + +# ============================================================================= +# Datadog Configuration (optional) +# ============================================================================= + +# Datadog API Key (for monitoring and logging) +DATADOG_API_KEY= + +# Datadog Application Key +DATADOG_APP_KEY= + +# Datadog Site (e.g., datadoghq.com, datadoghq.eu) +DATADOG_SITE=datadoghq.com + +# ============================================================================= +# Testing Configuration +# ============================================================================= + +# Set to "true" to enable test mode +TEST_MODE=false + +# Test database URL (separate from production) +TEST_DATABASE_URL=postgresql://postgres:postgres@localhost:5432/hermes_test?sslmode=disable + +# ============================================================================= +# Server Configuration +# ============================================================================= + +# Server address and port +SERVER_ADDRESS=:8000 + +# Log level (debug, info, warn, error) +LOG_LEVEL=info + +# Environment (development, staging, production) +ENVIRONMENT=development diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md new file mode 100644 index 000000000..527f20426 --- /dev/null +++ b/.github/copilot-instructions.md @@ -0,0 +1,311 @@ +# Hermes - GitHub Copilot Instructions + +## Project Overview +Hermes is an open-source document management system built by HashiCorp Labs. It's a full-stack application with a **Go backend** (1.25.0+) and an **Ember.js TypeScript frontend** (Node.js 20+), using PostgreSQL for data persistence and Algolia for search. The system integrates with Google Workspace for document storage and authentication. + +**Repository**: `hashicorp/hermes` | **Type**: Monorepo (Go + Ember.js) + +## Critical Build & Validation Workflow + +### 🔴 ALWAYS Follow This Sequence + +#### 1. **Backend (Go) - Build First** +```bash +# From repo root - REQUIRED order: +make bin # Build Go binary (fast, no web) +make go/test # Run Go tests (no DB required) +``` + +#### 2. **Frontend (Ember.js) - Then Validate** +```bash +cd web +yarn install # ALWAYS run first after pulling/switching branches +yarn test:types # TypeScript check (fast, catches type errors) +yarn lint:hbs # Template linting (533 files) +yarn build # Production build with env var warnings (expected) +``` + +#### 3. **Full Integration Build** +```bash +# From repo root: +make build # Builds web (yarn install + build) then Go binary +``` + + +### ⚠️ Known Build Behaviors & Workarounds + +**Web Build Environment Variables**: The web build shows ~10 env var warnings for optional configuration. This is **expected** and **not an error** - defaults are applied. Note: As of October 2025, `HERMES_WEB_ALGOLIA_APP_ID` and `HERMES_WEB_ALGOLIA_SEARCH_API_KEY` are **no longer needed** (search proxies through backend). `HERMES_WEB_GOOGLE_OAUTH2_CLIENT_ID` is **optional** (only needed for Google auth provider). + + +See `docs-internal/testing/` for detailed test analysis and fix strategies. + +**Linting Has Known Issues**: `yarn lint:js` reports 43 ESLint errors (mostly `@typescript-eslint/no-empty-object-type`). These are **non-blocking** - linting does not prevent builds or deployment. + +**Yarn Version**: The project uses **Yarn 4.10.3** (Berry/v3+). Commands like `yarn install --check-files` (from package.json `test:deps`) fail with newer Yarn. Use `yarn install --check-cache` instead or skip `yarn validate`. + +**PostgreSQL Container**: If `make docker/postgres/start` results in a restarting container, run: +```bash +make docker/postgres/stop && make docker/postgres/start +``` + +## Project Structure & Key Files + +### Root Level Configuration +- **`Makefile`**: Primary build orchestration (25+ targets) +- **`go.mod`**: Go 1.25.0, main deps: gorm, google.golang.org/api, algolia, datadog +- **`testing/`**: Complete containerized testing environment (see `testing/README.md`) +- **`config.hcl`**: **Fully documented runtime config** (tracked in git, 828 lines with comprehensive examples) +- **`configs/config.hcl`**: Minimal config template (246 lines, for reference only) +- **`testing/dex-config.yaml`**: Dex OIDC provider configuration for testing environment +- **`.gitignore`**: Excludes `credentials.json`, `token.json`, but NOT `config.hcl` + +### Backend Structure (`cmd/`, `internal/`, `pkg/`) +- **`cmd/hermes/main.go`**: Entry point +- **`internal/cmd/`**: CLI commands (server, indexer, operator) +- **`internal/api/`**: REST API handlers (v1 + v2) +- **`internal/server/`**: HTTP server setup +- **`internal/db/`**: PostgreSQL/GORM database layer +- **`pkg/models/`**: GORM models with extensive tests +- **`pkg/workspace/`**: Workspace management, adapters for Google and Local +- **`pkg/search/`**: Search abstraction, Algolia and Meilisearch adapters +- **`pkg/hashicorpdocs/`**: Document type handlers (RFC, PRD, FRD) + +### Frontend Structure (`web/`) +- **`web/package.json`**: Ember 6.7.0, TypeScript, Tailwind CSS, HDS components +- **`web/tsconfig.json`**: TypeScript with Ember paths, Glint for templates +- **`web/.eslintrc.js`**: TypeScript-ESLint with many rules disabled +- **`web/ember-cli-build.js`**: PostCSS/SASS + Tailwind build pipeline +- **`web/app/`**: Ember app source (components, routes, services) +- **`web/tests/`**: Acceptance & integration tests (currently broken) +- **`web/mirage/`**: API mocking for development + +## Continuous Integration (GitHub Actions) + +**Workflow**: `.github/workflows/ci.yml` (runs on PRs and main branch pushes) + +## Code Patterns & Standards + +### Go Conventions +- **CGO disabled**: All builds use `CGO_ENABLED=0` +- **Error handling**: Use `hashicorp/go-multierror` for aggregation +- **Logging**: `hashicorp/go-hclog` structured logging +- **Database**: GORM v2 with PostgreSQL driver, auto-migration on startup +- **Testing**: Table-driven tests, use `internal/test/database.go` for DB setup +- **Config**: HCL format via `hashicorp/hcl/v2` + +### TypeScript/Ember Conventions +- **Template syntax**: Ember strict mode with `