Add PostgreSQL WAL listener component

[emsearcy]

Adds a PostgreSQL WAL listener component to the Helm chart for real-time database change streaming to NATS.

The WAL listener monitors PostgreSQL tables and publishes change events to NATS, complementing the existing Meltano ETL pipeline with real-time data synchronization capabilities.

🤖 Generated with GitHub Copilot (via Zed)

Issue: LFXV2-991

[Copilot]

Pull request overview

This PR adds a PostgreSQL WAL (Write-Ahead Log) listener component to enable real-time database change streaming to NATS, complementing the existing Meltano ETL pipeline. The changes include significant refactoring of the sync helper service to support deletion operations, improved error handling with retry logic, and restructured Helm charts to accommodate multiple components.

Changes:

• Adds PostgreSQL WAL listener deployment with configurable filtering and NATS publishing
• Implements deletion handling (both hard and soft deletes) with tombstoning for mappings
• Refactors consumer patterns from manual fetch loops to the JetStream Consume API with proper error handling
• Updates dependencies across Go modules and Python packages

Reviewed changes

Copilot reviewed 29 out of 32 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
go.mod, go.sum	Updates Go dependencies to newer versions
uv.lock, pyproject.toml	Updates Python dependencies including Meltano 4.0.6→4.0.8
meltano/meltano.yml	Adds default incremental replication configuration
meltano/load/target-nats-kv/uv.lock	Updates target plugin dependencies
cmd/lfx-v1-sync-helper/main.go	Refactors consumer patterns and adds WAL listener subscription
cmd/lfx-v1-sync-helper/kv_watcher.go	Extracts KV message handling logic into separate file
cmd/lfx-v1-sync-helper/ingest_wal.go	New file implementing WAL event processing
cmd/lfx-v1-sync-helper/handlers*.go	Adds deletion handlers and retry logic
cmd/lfx-v1-sync-helper/config.go	Adds boolean environment variable parsing and UseMsgpack config
cmd/lfx-v1-sync-helper/client_*.go	Adds delete operations for projects and committees
charts/lfx-v1-sync-helper/templates/*.yaml	Restructures templates for multi-component deployment
charts/lfx-v1-sync-helper/values.yaml	Reorganizes values for app, WAL listener, and Meltano components
manifests/*.yaml	Removes standalone manifests (replaced by Helm chart)
README.md, AGENTS.md	Updates documentation for new components

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The function duplicates error detection logic that already exists in handlers_users.go (used in handleAlternateEmailUpdate). Consider extracting this to a shared utility function to avoid code duplication and ensure consistent error handling across the codebase.

[Copilot]

The database host is hardcoded to "localhost" and credentials use placeholder values in the config file. While these are overridden by environment variables, this configuration could cause confusion or be accidentally used. Consider using template variables or adding clear comments explaining the values are overridden.

Suggested change

	host: localhost
	port: {{ .Values.walListener.config.database.port }}
	name: {{ .Values.walListener.config.database.name | quote }}
	user: placeholder
	password: placeholder
	# NOTE: The database connection details below are placeholders and are
	# overridden by environment variables; these values are not used in production.
	host: "OVERRIDDEN_BY_ENV"
	port: {{ .Values.walListener.config.database.port }}
	name: {{ .Values.walListener.config.database.name | quote }}
	user: "OVERRIDDEN_BY_ENV"
	password: "OVERRIDDEN_BY_ENV"