Package System for Centralized Configuration Management #64
Description
Summary
Implement a package system that allows organizations to share and version standardized LHP configurations (presets, templates, substitutions) across multiple projects - similar to how dbt handles packages.
Motivation
Organizations with multiple LHP projects currently face:
- Configuration drift - each project evolves independently
- Duplication - same templates/presets copied across projects
- Maintenance overhead - updates must be manually propagated
- No version control - no way to track which config version each project uses
Proposed Solution
Package Installation
lhp.yaml
packages:
-
git: "https://github.com/acme-corp/lhp-central.git"
revision: "v1.0.0"
subdirectory: "packages/acme-standards" # Monorepo support -
local: "../shared-configs" # For development### Using Package Components
pipelines/customers.yaml
presets: [acme-standards.bronze_layer] # Qualified name
use_template: acme-standards.csv_ingestion### CLI Commands
lhp packages # Install from lhp.yaml
lhp packages --upgrade # Re-resolve versions
lhp packages --frozen # CI mode: fail if lock would change
lhp list-packages # Show installed packages### Lock File
lhp-package.lock.yaml ensures reproducible builds by recording exact Git commits.
Implementation Stages
| Stage | Description | Effort |
|---|---|---|
| 0 | Data models (PackageDependency, PackageLock, etc.) | Low |
| 1 | ResourceResolver for multi-source resolution | Medium |
| 2 | Parse packages from lhp.yaml |
Low |
| 3 | Lock file manager | Low |
| 4 | Git package downloader & installer | Medium |
| 5 | CLI commands | Medium |
Key Features
- Design document complete
- Stage 0: Data models
- Stage 1: Resource resolver
- Stage 2: Config parsing
- Stage 3: Lock file support
- Stage 4: Git downloader
- Stage 5: CLI commands
Design Document
See docs/LHP_PACKAGING_SYSTEM_DESIGN.md for full implementation details.
Acceptance Criteria
- Can install packages from Git repositories
- Can install packages from local paths (symlinks)
- Subdirectory support for monorepos
- Lock file generated and respected
- Qualified names work (
package.preset_name) - Local configs override package configs
-
--frozenflag fails CI if lock would change - Backward compatible (existing projects work without packages)
Related
- Inspired by: dbt packages, Terraform modules, Helm charts## Summary
Implement a package system that allows organizations to share and version standardized LHP configurations (presets, templates, substitutions) across multiple projects - similar to how dbt handles packages.
Motivation
Organizations with multiple LHP projects currently face:
- Configuration drift - each project evolves independently
- Duplication - same templates/presets copied across projects
- Maintenance overhead - updates must be manually propagated
- No version control - no way to track which config version each project uses
Proposed Solution
Package Installation
lhp.yaml
packages:
-
git: "https://github.com/acme-corp/lhp-central.git"
revision: "v1.0.0"
subdirectory: "packages/acme-standards" # Monorepo support -
local: "../shared-configs" # For development### Using Package Components
pipelines/customers.yaml
presets: [acme-standards.bronze_layer] # Qualified name
use_template: acme-standards.csv_ingestion### CLI Commands
lhp packages # Install from lhp.yaml
lhp packages --upgrade # Re-resolve versions
lhp packages --frozen # CI mode: fail if lock would change
lhp list-packages # Show installed packages### Lock File
lhp-package.lock.yaml ensures reproducible builds by recording exact Git commits.
Implementation Stages
| Stage | Description | Effort |
|---|---|---|
| 0 | Data models (PackageDependency, PackageLock, etc.) | Low |
| 1 | ResourceResolver for multi-source resolution | Medium |
| 2 | Parse packages from lhp.yaml |
Low |
| 3 | Lock file manager | Low |
| 4 | Git package downloader & installer | Medium |
| 5 | CLI commands | Medium |
Key Features
- Design document complete
- Stage 0: Data models
- Stage 1: Resource resolver
- Stage 2: Config parsing
- Stage 3: Lock file support
- Stage 4: Git downloader
- Stage 5: CLI commands
Design Document
See docs/LHP_PACKAGING_SYSTEM_DESIGN.md for full implementation details.
Acceptance Criteria
- Can install packages from Git repositories
- Can install packages from local paths (symlinks)
- Subdirectory support for monorepos
- Lock file generated and respected
- Qualified names work (
package.preset_name) - Local configs override package configs
-
--frozenflag fails CI if lock would change - Backward compatible (existing projects work without packages)
Related
- Inspired by: dbt packages, Terraform modules, Helm charts