Skip to content

Commit 1ae5634

Browse files
avivsinaiavivsinai
committed
fix: improve CI test configuration without version freezing
- Add explicit CI and PROMPTCODE_TEST env vars - Use --bail=1 to stop at first failure for easier debugging - Reduce test timeout to 5s (from 10s) - Keep using latest versions to catch issues early
1 parent 08c8283 commit 1ae5634

File tree

9 files changed

+556
-4
lines changed

9 files changed

+556
-4
lines changed

‎.claude/commands/promptcode-ask-expert.md‎

Lines changed: 161 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
---
2+
allowed-tools: Bash(promptcode expert:*), Bash(promptcode preset list:*), Bash(promptcode generate:*), Bash(open -a Cursor:*), Read(/tmp/expert-*:*), Write(/tmp/expert-consultation-*.md), Task
3+
description: Consult AI expert (O3/O3-pro) for complex problems with code context - supports ensemble mode for multiple models
4+
---
5+
6+
Consult an expert about: $ARGUMENTS
7+
8+
## Instructions:
9+
10+
1. Analyze the request in $ARGUMENTS:
11+
- Extract the main question/problem
12+
- Identify if code context would help (look for keywords matching our presets)
13+
- Check for multiple model requests (e.g., "compare using o3 and gpt-5", "ask o3, gpt-5, and gemini")
14+
- Available models from our MODELS list: o3, o3-pro, o3-mini, gpt-5, gpt-5-mini, gpt-5-nano, sonnet-4, opus-4, gemini-2.5-pro, gemini-2.5-flash, grok-4
15+
- If 2+ models detected → use ensemble mode
16+
- For single model: determine preference (if user mentions "o3-pro" or "o3 pro", use o3-pro)
17+
18+
2. If code context needed, list available presets:
19+
```bash
20+
promptcode preset list
21+
```
22+
Choose relevant preset(s) based on the question.
23+
24+
3. Prepare consultation file for review:
25+
- Create a consultation file at `/tmp/expert-consultation-{timestamp}.md`
26+
- Structure the file with:
27+
```markdown
28+
# Expert Consultation
29+
30+
## Question
31+
{user's question}
32+
33+
## Context
34+
{any relevant context or background}
35+
```
36+
- If a preset would help, append the code context:
37+
```bash
38+
echo -e "\n## Code Context\n" >> "/tmp/expert-consultation-{timestamp}.md"
39+
promptcode generate --preset "{preset_name}" >> "/tmp/expert-consultation-{timestamp}.md"
40+
```
41+
42+
4. Open consultation for user review (if Cursor is available):
43+
```bash
44+
open -a Cursor "/tmp/expert-consultation-{timestamp}.md"
45+
```
46+
47+
5. Estimate cost and get approval:
48+
- Model costs (from our pricing):
49+
- O3: $2/$8 per million tokens (input/output)
50+
- O3-pro: $20/$80 per million tokens (input/output)
51+
- GPT-5: $1.25/$10 per million tokens
52+
- GPT-5-mini: $0.25/$2 per million tokens
53+
- Sonnet-4: $5/$20 per million tokens
54+
- Opus-4: $25/$100 per million tokens
55+
- Gemini-2.5-pro: $3/$12 per million tokens
56+
- Grok-4: $5/$15 per million tokens
57+
- Calculate based on file size (roughly: file_size_bytes / 4 = tokens)
58+
59+
**For single model:**
60+
- Say: "I've prepared the expert consultation (~{tokens} tokens). Model: {model}. You can edit the file to refine your question. Reply 'yes' to send to the expert (estimated cost: ${cost})."
61+
62+
**For ensemble mode (multiple models):**
63+
- Calculate total cost across all models
64+
- Say: "I've prepared an ensemble consultation (~{tokens} tokens) with {models}. Total estimated cost: ${total_cost} ({model1}: ${cost1}, {model2}: ${cost2}, ...). Reply 'yes' to proceed with all models in parallel."
65+
66+
6. Execute based on mode:
67+
68+
**Single Model Mode:**
69+
```bash
70+
promptcode expert --prompt-file "/tmp/expert-consultation-{timestamp}.md" --model {model} --yes
71+
```
72+
73+
**Ensemble Mode (Parallel Execution):**
74+
- Use Task tool to run multiple models in parallel
75+
- Each task runs the same consultation file with different models
76+
- Store each result in separate file: `/tmp/expert-{model}-{timestamp}.txt`
77+
- Example for 3 models (run these in PARALLEL using Task tool):
78+
```
79+
Task 1: promptcode expert --prompt-file "/tmp/expert-consultation-{timestamp}.md" --model o3 --yes > /tmp/expert-o3-{timestamp}.txt
80+
Task 2: promptcode expert --prompt-file "/tmp/expert-consultation-{timestamp}.md" --model gpt-5 --yes > /tmp/expert-gpt5-{timestamp}.txt
81+
Task 3: promptcode expert --prompt-file "/tmp/expert-consultation-{timestamp}.md" --model gemini-2.5-pro --yes > /tmp/expert-gemini-{timestamp}.txt
82+
```
83+
- IMPORTANT: Launch all tasks at once for true parallel execution
84+
- Wait for all tasks to complete
85+
- Note: The --yes flag confirms we have user approval for the cost
86+
87+
7. Handle the response:
88+
89+
**Single Model Mode:**
90+
- If successful: Open response in Cursor (if available) and summarize key insights
91+
- If API key missing: Show appropriate setup instructions
92+
93+
**Ensemble Mode (Synthesis):**
94+
- Read all response text files
95+
- Extract key insights from each model's response
96+
- Create synthesis report in `/tmp/expert-ensemble-synthesis-{timestamp}.md`:
97+
98+
```markdown
99+
# Ensemble Expert Consultation Results
100+
101+
## Question
102+
{original_question}
103+
104+
## Expert Responses
105+
106+
### {Model1} - ${actual_cost}, {response_time}s
107+
**Key Points:**
108+
- {key_point_1}
109+
- {key_point_2}
110+
- {key_point_3}
111+
112+
### {Model2} - ${actual_cost}, {response_time}s
113+
**Key Points:**
114+
- {key_point_1}
115+
- {key_point_2}
116+
- {key_point_3}
117+
118+
## Synthesis
119+
120+
**Consensus Points:**
121+
- {point_agreed_by_multiple_models}
122+
- {another_consensus_point}
123+
124+
**Best Comprehensive Answer:** {Model} provided the most thorough analysis, particularly strong on {specific_aspect}
125+
126+
**Unique Insights:**
127+
- {Model1}: {unique_insight_from_model1}
128+
- {Model2}: {unique_insight_from_model2}
129+
130+
**🏆 WINNER:** {winning_model} - {clear_reason_why_this_model_won}
131+
(If tie: "TIE - Both models provided equally valuable but complementary insights")
132+
133+
**Performance Summary:**
134+
- Total Cost: ${total_actual_cost}
135+
- Total Time: {total_time}s
136+
- Best Value: {model_with_best_cost_to_quality_ratio}
137+
```
138+
139+
- Open synthesis in Cursor if available
140+
- IMPORTANT: Always declare a clear winner (or explicitly state if it's a tie)
141+
- Provide brief summary of which model performed best and why they won
142+
143+
**Error Handling:**
144+
- If any model fails in ensemble mode, continue with successful ones
145+
- Report which models succeeded/failed
146+
- If OPENAI_API_KEY missing:
147+
```
148+
To use expert consultation, set your OpenAI API key:
149+
export OPENAI_API_KEY=sk-...
150+
Get your key from: https://platform.openai.com/api-keys
151+
```
152+
- For other errors: Report exact error message
153+
154+
## Important:
155+
- Default to O3 model unless O3-pro explicitly requested or needed for complex reasoning
156+
- For ensemble mode: limit to maximum 4 models to prevent resource exhaustion
157+
- Always show cost estimate before sending
158+
- Keep questions clear and specific
159+
- Include relevant code context when asking about specific functionality
160+
- NEVER automatically add --yes without user approval
161+
- Reasoning effort defaults to 'high' (set in CLI) - no need to specify

‎.claude/commands/promptcode-preset-create.md‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
---
2+
allowed-tools: Bash(promptcode preset create:*), Bash(promptcode preset info:*), Glob(**/*), Grep, Write(.promptcode/presets/*.patterns)
3+
description: Create a promptcode preset from description
4+
---
5+
6+
Create a promptcode preset for: $ARGUMENTS
7+
8+
## Instructions:
9+
10+
1. Parse the description to understand what code to capture:
11+
- Look for keywords like package names, features, components, integrations
12+
- Identify if it's Python, TypeScript, or mixed code
13+
- Determine the scope (single package, cross-package feature, etc.)
14+
15+
2. Research the codebase structure:
16+
- Use Glob to explore relevant directories
17+
- Use Grep to find related files if needed
18+
- Identify the main code locations and any related tests/docs
19+
20+
3. Generate a descriptive preset name:
21+
- Use kebab-case (e.g., "auth-system", "microlearning-utils")
22+
- Keep it concise but descriptive
23+
24+
4. Create the preset:
25+
```bash
26+
promptcode preset create "{preset_name}"
27+
```
28+
This creates `.promptcode/presets/{preset_name}.patterns`
29+
30+
5. Edit the preset file to add patterns:
31+
- Start with a header comment explaining what the preset captures
32+
- Add inclusion patterns for the main code
33+
- Add patterns for related tests and documentation
34+
- Include common exclusion patterns:
35+
- `!**/__pycache__/**`
36+
- `!**/*.pyc`
37+
- `!**/node_modules/**`
38+
- `!**/dist/**`
39+
- `!**/build/**`
40+
41+
6. Test and report results:
42+
```bash
43+
promptcode preset info "{preset_name}"
44+
```
45+
Report the file count and estimated tokens.
46+
47+
## Common Pattern Examples:
48+
- Python package: `python/cogflows-py/packages/{package}/src/**/*.py`
49+
- TypeScript component: `ts/next/{site}/components/{component}/**/*.{ts,tsx}`
50+
- Cross-package feature: Multiple specific paths
51+
- Tests: `python/cogflows-py/packages/{package}/tests/**/*.py`
52+
- Documentation: `**/{feature}/**/*.md`

‎.claude/commands/promptcode-preset-info.md‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
---
2+
allowed-tools: Bash(promptcode preset info:*), Bash(promptcode preset list:*), Glob(.promptcode/presets/*.patterns), Read(.promptcode/presets/*.patterns:*)
3+
description: Show detailed information about a promptcode preset
4+
---
5+
6+
Show detailed information about promptcode preset: $ARGUMENTS
7+
8+
## Instructions:
9+
10+
1. Parse the arguments to identify the preset:
11+
- If exact preset name provided (e.g., "functional-framework"), use it directly
12+
- If description provided, infer the best matching preset:
13+
- Run `promptcode preset list` to see available presets
14+
- Read header comments from preset files in `.promptcode/presets/` if needed
15+
- Match based on keywords and context
16+
- Choose the most relevant preset
17+
18+
2. Run the promptcode info command with the determined preset name:
19+
```bash
20+
promptcode preset info "{preset_name}"
21+
```
22+
23+
3. If a preset was inferred from description, explain which preset was chosen and why.
24+
25+
The output will show:
26+
- Preset name and path
27+
- Description from header comments
28+
- File count and token statistics
29+
- Pattern details
30+
- Sample files included
31+
- Usage instructions

‎.claude/commands/promptcode-preset-list.md‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
---
2+
allowed-tools: Bash(promptcode preset list:*)
3+
description: List all available promptcode presets with pattern counts
4+
---
5+
6+
List all available promptcode presets.
7+
8+
Run the command:
9+
```bash
10+
promptcode preset list
11+
```
12+
13+
This will display all available presets with their pattern counts. Use the preset names with other promptcode commands to work with specific code contexts.

‎.claude/commands/promptcode-preset-to-prompt.md‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
---
2+
allowed-tools: Bash(promptcode generate:*), Bash(promptcode preset list:*), Glob(.promptcode/presets/*.patterns), Read(.promptcode/presets/*.patterns:*)
3+
description: Generate AI-ready prompt file from a promptcode preset
4+
---
5+
6+
Generate prompt file from promptcode preset: $ARGUMENTS
7+
8+
## Instructions:
9+
10+
1. Parse arguments to understand what the user wants:
11+
- Extract preset name or description
12+
- Extract output path/filename if specified (e.g., "to ~/Desktop/analysis.txt", "in /tmp/", "as myfile.txt")
13+
14+
2. If inferring from description:
15+
- Run `promptcode preset list` to see available presets
16+
- Read header comments from `.promptcode/presets/*.patterns` files if needed
17+
- Match based on keywords and context
18+
- Choose the most relevant preset
19+
20+
3. Determine output path:
21+
- Default: `/tmp/promptcode-{preset-name}-{timestamp}.txt` where timestamp is YYYYMMDD-HHMMSS
22+
- If user specified just a folder: `{folder}/promptcode-{preset-name}-{timestamp}.txt`
23+
- If user specified filename without path: `/tmp/{filename}`
24+
- If user specified full path: use exactly as specified
25+
26+
4. Generate the prompt file:
27+
```bash
28+
promptcode generate --preset "{preset_name}" --output "{output_path}"
29+
```
30+
31+
5. Report results:
32+
- Which preset was used (especially important if inferred)
33+
- Full path to the output file
34+
- Token count and number of files included
35+
- Suggest next steps (e.g., "You can now open this file in your editor")
36+
37+
## Examples of how users might call this:
38+
- `/promptcode-preset-to-prompt functional-framework`
39+
- `/promptcode-preset-to-prompt microlearning analysis to ~/Desktop/`
40+
- `/promptcode-preset-to-prompt the functional code as analysis.txt`

‎.github/workflows/release.yml‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ on:
88

99
jobs:
1010
test:
11-
runs-on: ubuntu-20.04 # Use stable runner version
11+
runs-on: ubuntu-24.04
1212
timeout-minutes: 10 # Add timeout to prevent hanging
1313
steps:
1414
- uses: actions/checkout@v4
@@ -31,16 +31,17 @@ jobs:
3131

3232
- name: Setup Bun for CLI tests
3333
uses: oven-sh/setup-bun@v2
34-
with:
35-
bun-version: 1.0.4 # Pin to known stable version
3634

3735
- name: CLI tests
3836
timeout-minutes: 5 # Add step-level timeout
37+
env:
38+
CI: true
39+
PROMPTCODE_TEST: 1
3940
run: |
4041
cd packages/cli
4142
bun install --frozen-lockfile
4243
bun run build
43-
bun test --timeout 10000 # Increase per-test timeout to 10s
44+
PROMPTCODE_TEST=1 CI=true bun test --bail=1 --timeout 5000
4445
4546
build-extension:
4647
needs: test

‎.promptcode/presets/domain-boundaries.patterns‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# Domain boundaries analysis preset
2+
# Analyzing separation between CLI layer and agent command layers
3+
# Created: 2025-01-13
4+
5+
# CLI Layer - Core functionality
6+
packages/cli/src/commands/expert.ts
7+
packages/cli/src/commands/cc.ts
8+
packages/cli/src/commands/cursor.ts
9+
packages/cli/src/utils/cost.ts
10+
packages/cli/src/utils/environment.ts
11+
packages/cli/src/providers/models.ts
12+
packages/cli/src/providers/ai-provider.ts
13+
14+
# Agent Command Layer - Claude Code
15+
.claude/commands/*.md
16+
17+
# Agent Templates
18+
packages/cli/src/claude-templates/*.md
19+
packages/cli/src/cursor-templates/*.mdc
20+
21+
# Integration utilities
22+
packages/cli/src/utils/claude-integration.ts
23+
packages/cli/src/utils/cursor-integration.ts
24+
packages/cli/src/utils/integration-helper.ts
25+
26+
# Exclude test files and build artifacts
27+
!**/*.test.ts
28+
!**/dist/**
29+
!**/node_modules/**

0 commit comments

Comments
 (0)