Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
a3441e2
Add performance timing instrumentation
e-gineer Dec 28, 2025
8a6605b
Add comprehensive mod loading tests for workspace
e-gineer Dec 28, 2025
4c880fd
Add performance benchmark tests for mod loading and payload building
e-gineer Dec 28, 2025
409af0f
Add baseline performance measurements and project workflow
e-gineer Dec 28, 2025
2d5c71e
Update Task 5 status: Parallelize File I/O complete
e-gineer Dec 28, 2025
5c0668a
Update Task 6 status: Parallelize HCL Parsing complete
e-gineer Dec 28, 2025
0c1fdd2
Optimize database client creation with concurrent initialization
e-gineer Dec 28, 2025
111fcf8
Enable local pipe-fittings for getSourceDefinition optimization
e-gineer Dec 28, 2025
4319b1d
Complete Task 9: Final performance validation
e-gineer Dec 29, 2025
2c8d342
Add lazy loading workspace for dashboard server
e-gineer Dec 29, 2025
e2a9363
Fix missing mod_full_name and trunks in lazy loading payload
e-gineer Dec 29, 2025
e16e318
Implement hybrid lazy loading for dashboard server execution
e-gineer Dec 30, 2025
df6be51
Add comprehensive test suite for lazy loading
e-gineer Jan 2, 2026
a84c99a
Add HCL syntax scanner for resource indexing
e-gineer Jan 2, 2026
9519451
Consolidate scanner to use HCL parser exclusively
e-gineer Jan 2, 2026
143f8c0
Enable lazy loading by default
e-gineer Jan 2, 2026
a7d0a91
Update integration tests for lazy loading by default
e-gineer Jan 2, 2026
c06ad5b
Clean up PR: remove dev utilities and update pipe-fittings reference
e-gineer Jan 2, 2026
18684a9
Remove .claude/wip/ from git tracking
e-gineer Jan 2, 2026
904e874
Run go fmt and remove orphaned benchmark tests
e-gineer Jan 2, 2026
7522a45
Fix race detector test: skip timing assertion under race
e-gineer Jan 2, 2026
8a923d4
fix: address CI linting and test failures
e-gineer Jan 2, 2026
7ad16be
Fix remaining linting errors for PR #990
e-gineer Jan 2, 2026
69029e2
Fix remaining file permission issues (gosec G306)
e-gineer Jan 2, 2026
0d6e206
Fix workspace tests to use committed test data
e-gineer Jan 2, 2026
fa16a24
Fix workspace tests to use committed lazy-loading-tests/generated data
e-gineer Jan 2, 2026
0325baf
Fix generated test data: add missing nested_level_1 benchmark
e-gineer Jan 2, 2026
c07da86
Increase CI test timeout from 30s to 300s
e-gineer Jan 2, 2026
dbacc71
Fix source_definition not populated for resource show commands
e-gineer Jan 2, 2026
ae896b8
Fix CLI integration tests for JSON output format
e-gineer Jan 5, 2026
3811e9f
Fix CI acceptance test exit code capture
e-gineer Jan 5, 2026
1ff74df
Fix CI test exit code capture with set -e
e-gineer Jan 5, 2026
5cf9650
Fix lazy loading scanner to include .sp files
e-gineer Jan 5, 2026
4a5e77c
Fix lazy loading to only apply to benchmark commands
e-gineer Jan 5, 2026
7098e47
Fix lazy loading to disable when tag/where filters are used
e-gineer Jan 5, 2026
05c8579
docs: Add local testing instructions to CLAUDE.md
e-gineer Jan 5, 2026
07c6ce2
Add verify skill for local testing workflow
e-gineer Jan 5, 2026
6780f00
Update pipe-fittings to fix JSON serialization of embedded structs
e-gineer Jan 5, 2026
e370406
Disable lazy loading when dependency mods are installed
e-gineer Jan 5, 2026
83da632
Revert "Disable lazy loading when dependency mods are installed"
e-gineer Jan 6, 2026
7543d8f
Fix lazy loading scanner to handle mod.sp files and cross-mod references
e-gineer Jan 6, 2026
fa4ce9c
Fix lazy loading to resolve sql = query.xxx.sql references
e-gineer Jan 6, 2026
fe1afc9
Detect duplicate mod versions and fall back to eager loading
e-gineer Jan 6, 2026
2bbc833
Add fallback to eager loading in init_data for duplicate mod versions
e-gineer Jan 6, 2026
7c894b7
Enable lazy loading for dashboard server command
e-gineer Jan 6, 2026
2994af5
Implement phased HCL loading for fast startup with complete metadata
e-gineer Jan 7, 2026
38008e9
Skip performance tests in short mode
e-gineer Jan 7, 2026
f90f9df
Skip comparison tests when generated testdata is missing
e-gineer Jan 7, 2026
c83d16e
Skip dashboardserver comparison tests when generated testdata is missing
e-gineer Jan 7, 2026
3a232df
Fix lazy loading target resolution for dashboards
e-gineer Jan 7, 2026
fe7301f
Force eager loading in CI acceptance tests
e-gineer Jan 7, 2026
7681725
Restore full CI test coverage and enable lazy loading tests
e-gineer Jan 7, 2026
c404a42
Fix server integration tests to use DuckDB instead of Steampipe
e-gineer Jan 7, 2026
6bf1bb4
Skip server integration tests in CI (require dashboard assets)
e-gineer Jan 7, 2026
61695db
Fix CI test failures
e-gineer Jan 7, 2026
3ee91cc
Fix anonymous block naming to match pipe-fittings convention
e-gineer Jan 7, 2026
8949965
Consolidate duplicate code with pipe-fittings for lazy loading
e-gineer Jan 7, 2026
f25c4dd
Consolidate scanner to single file, remove race build tags
e-gineer Jan 7, 2026
fa9e2a6
Fix CI test failures for lazy loading
e-gineer Jan 7, 2026
8eaede8
Refactor base reference resolution to use unified eval context approach
e-gineer Jan 7, 2026
4502b25
Add lazy loading for list commands (21x faster)
e-gineer Jan 7, 2026
7c6bbcc
Update pipe-fittings dependency to v2.8.0-rc.0
pskrbasu Jan 13, 2026
9165100
Fix missing resource_name field in lazy list JSON output
pskrbasu Jan 13, 2026
00da8e5
Make benchmark tests self-contained for CI
e-gineer Jan 23, 2026
3b69594
Add missing fields to lazy list JSON output for parity with eager mode
e-gineer Jan 23, 2026
d649263
Add nolint directives for MD5 usage in resource hash generation
e-gineer Jan 23, 2026
855e8f9
Add control list JSON fields and full path computation for lazy loading
e-gineer Jan 23, 2026
b062e67
Wait for background resolution before list output when entries have u…
e-gineer Jan 23, 2026
bd40af5
Start background resolution when lazy workspace is created
e-gineer Jan 23, 2026
ddc0f81
Skip variable tags test as known lazy loading limitation
e-gineer Jan 23, 2026
acccddb
Add variable and locals resolution for lazy loading tags
e-gineer Jan 23, 2026
b1bb368
Use full qualified name in list output and add tests for format verif…
pskrbasu Jan 27, 2026
aaef908
Update pipe-fittings dependency to v2.8.0-rc.1
pskrbasu Jan 27, 2026
9035ead
Add fallback to eager loading when lazy index is empty
pskrbasu Jan 29, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 71 additions & 0 deletions .claude/CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Powerpipe

Powerpipe is a CLI tool for dashboards, benchmarks, and compliance monitoring. Written in Go with a React dashboard UI.

## Build Commands

```bash
make build # Build Go binary to /usr/local/bin
make dashboard_assets # Build React dashboard
make all # Build both
```

## Testing

```bash
go test ./... # Run Go unit tests
```

Use `/verify` skill for full local verification workflow including acceptance tests.

## Project Structure

- `internal/cmd/` - CLI commands (check, dashboard, query, etc.)
- `internal/controldisplay/` - Control output formatting
- `internal/controlexecute/` - Control execution logic
- `internal/workspace/` - Workspace loading (lazy and eager modes)
- `internal/resourceindex/` - Fast HCL metadata extraction
- `internal/dashboardserver/` - Dashboard server and WebSocket handling
- `ui/dashboard/` - React dashboard application

## Workspace Loading

Powerpipe uses phased loading to balance fast startup with complete functionality.

### Loading Phases

1. **Phase 1 (Index Build)**: Parse HCL syntax, extract metadata
- Fast: ~300-500ms for large mods
- Captures: names, titles, descriptions, tags (literals)
- Skips: reference resolution, validation

2. **Phase 2 (Background Resolution)**: Resolve dynamic metadata
- Runs in background goroutines
- Resolves: variable references, templates, function calls
- Updates index progressively

3. **Phase 3 (On-Demand Loading)**: Full resource loading
- Triggered by user interaction (click dashboard, run benchmark)
- Full HCL parsing with reference resolution
- Results cached for reuse

### Configuration

| Environment Variable | Description | Default |
|---------------------|-------------|---------|
| `POWERPIPE_WORKSPACE_PRELOAD` | Force eager loading (true/false) | false |

### Key Files

- `internal/workspace/lazy_workspace.go` - Lazy workspace coordinator
- `internal/workspace/background_resolver.go` - Background resolution
- `internal/resourceindex/scanner_hcl.go` - HCL metadata extraction
- `internal/resourceindex/entry.go` - Index entry structure
- `internal/resourceloader/loader.go` - On-demand resource loading

## Skills

| Skill | When to Use |
|-------|-------------|
| `verify` | Local verification before pushing (build, lint, tests) |
| `project-workflow` | Complex multi-task projects, parallel execution |
224 changes: 224 additions & 0 deletions .claude/skills/project-workflow/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,224 @@
---
name: project-workflow
description: Plan and organize complex multi-task projects. Use for work requiring 3+ related tasks, parallel execution, or careful coordination. Creates .claude/wip/ project structure.
---

# Project Workflow Skill

This skill provides patterns for planning, organizing, and executing complex projects with multiple related tasks. Whether working sequentially or with parallel agents, this skill helps structure larger efforts effectively.

## Purpose

When working on projects with multiple related tasks (e.g., fixing several bugs, implementing a feature set, refactoring a subsystem), this skill helps:
- Plan and break down complex work into manageable tasks
- Organize work for sequential or parallel execution
- Maintain clear task boundaries and dependencies
- Track overall progress across tasks and agents
- Prevent merge conflicts using git worktrees (for parallel work)

## Core Workflow

### 1. Create Work Directory

```bash
mkdir -p .claude/wip/<topic-name>
```

Examples:
- `.claude/wip/bug-fixes-wave-1/`
- `.claude/wip/refactor-auth-layer/`
- `.claude/wip/oci-policy-additions/`

### 2. Create Project Plan and Task Files

Create individual task files and a coordination plan (whether working alone or with multiple agents):

```bash
# In .claude/wip/<topic>/
task-1-description.md # First task
task-2-description.md # Second task
task-3-description.md # Third task
plan.md # Overall project plan and dependencies
status.md # Progress tracking (optional)
```

#### Task File Format

Each `task-N-*.md` file should contain:

```markdown
# Task N: [Brief Title]

## Objective
Clear, specific goal for this task

## Context
- Why this task exists
- How it relates to other tasks
- Key constraints or requirements

## Dependencies
- Files/modules this task will modify
- Other tasks that must complete first (if any)
- Shared resources to coordinate

## Acceptance Criteria
- [ ] Criterion 1
- [ ] Criterion 2
- [ ] Tests pass

## Notes
Any additional context or considerations
```

#### Plan File Format

The `plan.md` should contain:

```markdown
# Project Plan: [Topic Name]

## Overview
High-level description of the project and its goals

## Task Breakdown
1. **Task 1**: Brief description (Status: Pending)
2. **Task 2**: Brief description (Status: Pending)
3. **Task 3**: Brief description (Status: Pending)

## Execution Strategy
- Sequential: Tasks run one after another
- Parallel: Multiple agents work simultaneously on independent tasks

## Dependencies
- Task 2 depends on Task 1 completing
- Task 3 can run in parallel with Task 1
- Task 4 requires Tasks 2 and 3 to be complete

## Shared Resources
- Files that multiple tasks might touch
- Coordination strategy for conflicts (if working in parallel)

## Integration Plan
How to merge work back together (especially for parallel work)

## Notes
Additional coordination details
```

### 3. Git Worktree Setup (Recommended for Parallel Work)

To prevent merge conflicts when agents work in parallel:

```bash
# Create worktree for each agent
git worktree add .claude/wip/<topic>/worktree-agent-1 -b wip/<topic>/task-1
git worktree add .claude/wip/<topic>/worktree-agent-2 -b wip/<topic>/task-2
git worktree add .claude/wip/<topic>/worktree-agent-3 -b wip/<topic>/task-3
```

Then agents work in their respective worktrees:
```bash
cd .claude/wip/<topic>/worktree-agent-1
# Agent 1 does work here

cd .claude/wip/<topic>/worktree-agent-2
# Agent 2 does work here
```

When done:
```bash
# Merge branches back to main
git checkout main
git merge wip/<topic>/task-1
git merge wip/<topic>/task-2

# Clean up worktrees
git worktree remove .claude/wip/<topic>/worktree-agent-1
git worktree remove .claude/wip/<topic>/worktree-agent-2
```

### 4. Executing Tasks

**Sequential execution** (work through tasks one at a time):
```
"Read plan in .claude/wip/bug-fixes/ and execute task-1"
[Wait for completion]
"Read plan in .claude/wip/bug-fixes/ and execute task-2"
[Wait for completion]
"Read plan in .claude/wip/bug-fixes/ and execute task-3"
```

**Parallel execution** (launch multiple agents simultaneously):
```
Launch 3 agents in parallel:
- Agent 1: "Read .claude/wip/bug-fixes/task-1-*.md and execute"
- Agent 2: "Read .claude/wip/bug-fixes/task-2-*.md and execute"
- Agent 3: "Read .claude/wip/bug-fixes/task-3-*.md and execute"
```

**Mixed approach** (some sequential, some parallel):
```
"Read and execute task-1" # Must complete first
[Wait for completion]

Launch tasks 2 and 3 in parallel: # Both depend on task 1
- Agent 1: "Read and execute task-2"
- Agent 2: "Read and execute task-3"
```

## When to Use This Pattern

**Use when:**
- Working on complex projects with 3+ related tasks
- Need to track progress across multiple work items
- Want to break down large efforts into manageable pieces
- Working with clear task boundaries and dependencies
- Want option to parallelize independent work for speed

**Don't use when:**
- Single straightforward task
- Tightly coupled changes requiring iterative exploration
- Unclear requirements (explore first, then create plan)
- Very small changes (overkill for simple edits)

## Tips for Effective Project Planning

1. **Clear boundaries**: Ensure tasks are well-defined with specific goals
2. **Document dependencies**: Be explicit about task ordering and relationships
3. **Track progress**: Use `status.md` or update task files as work completes
4. **Atomic tasks**: Each task should be independently testable
5. **Plan integration**: Know how pieces fit together before starting
6. **Choose execution mode**: Sequential for dependent work, parallel for independent tasks
7. **Use worktrees for parallel work**: Prevents git conflicts when multiple agents work simultaneously

## Example: Bug Fix Wave

```bash
# Setup
mkdir -p .claude/wip/bug-fixes-wave-1

# Create tasks (coordinator)
echo "Fix null pointer in auth.ts" > .claude/wip/bug-fixes-wave-1/task-1-auth-null.md
echo "Fix race condition in sync.ts" > .claude/wip/bug-fixes-wave-1/task-2-sync-race.md
echo "Fix memory leak in cache.ts" > .claude/wip/bug-fixes-wave-1/task-3-cache-leak.md

# Execute tasks (choose sequential or parallel)
# Sequential:
# "Execute .claude/wip/bug-fixes-wave-1/task-1-auth-null.md"

# Parallel:
# Launch 3 agents in parallel with task-1, task-2, task-3
```

## Cleanup

After all tasks complete and merge:

```bash
# Archive the work plan
mv .claude/wip/<topic> .claude/wip/archive/<topic>-$(date +%Y%m%d)

# Or just delete if not needed
rm -rf .claude/wip/<topic>
```
Loading
Loading