Skip to content

V2_MASTER_PLAN.md

Latest commit

 

History

History
189 lines (143 loc) · 5.64 KB

V2_MASTER_PLAN.md

File metadata and controls

189 lines (143 loc) · 5.64 KB
Raw

CodeWell V2 Master Plan

Last updated: 2026-06-14

Completed Phases

Phase 1: V1 Baseline Freeze

  • Preserved V1 open-source evaluation baseline
  • Identified strong_multi_goal task family as V2 focus
  • Established evaluation infrastructure (agent_eval_session, repeated_run_summary)

Phase 2: V2 Core Implementation

  • Ambiguity-aware retrieval (navigation_prune mode)
  • Structured subgoal decomposition and shallow probing
  • Shared-workspace clustering (path-prefix + relation-overlap)
  • Primary/backup branch selection with micro-context retention
  • Default-output trimming, stable-entrypoint anchoring
  • Repeated-run evaluation: 3 strong_multi_goal tasks, all net-positive
  • Status: Frozen — commit ac67177

Phase 2.5: Quality Hardening (7 Steps)

  1. Parser tests: +111 tests (Python AST + JS/TS regex)
  2. Store tests: +41 tests (SQLite engine, FTS, schema migrations)
  3. _now_utc dedup: 9 copies → shared codewell.utils.now_utc
  4. N+1 INSERT batching: executemany for symbols/imports/call_edges
  5. Exception handling audit + CHANGELOG stale claims fix
  6. Scanner/commands/repair_audit tests: +40 tests
  7. Dedupe unification + 10 core function docstrings
  • Tests: 295 → 487 (+65%)
  • Status: Complete — commit ac67177

Phase 3: Benchmark Expansion (5 New Domains)

  • Security/config, validation/schema, frontend auth, ORM/database, framework config
  • 3×3 repeated-run protocol: 30 runs total
  • Initial results: 3/5 elapsed-positive, 5/5 tools-positive
  • Post-fix (security + pipe keyword expansion): 5/5 all-positive
  • Status: Complete — commits 9de45bd, 5e61f29

Phase 3.5: Merge to Main

  • Merged feature branch to main with --no-ff
  • Tagged v2.0.0-alpha
  • Status: Complete — commit f61dc3c

Current Phase: Phase 4 — More New Domains

Tasks (3 tasks, 3 new domains)

# Task Domain Goal Shape
1 nestjs_file_upload_type_validation_wiring File handling weak_multi_goal
2 nestjs_mail_i18n_context_null_safety_wiring Email/notifications strong_multi_goal
3 nestjs_serializer_interceptor_global_wiring Data serialization weak_multi_goal

Protocol

  • 3 tasks × 3 repeats × 2 conditions = 18 runs
  • Expected duration: ~45-90 minutes

Success Criteria

  • No regression in existing 11 tasks
  • At least 2/3 new tasks elapsed-positive
  • Tools-positive on all 3 new tasks

Status: Ready to execute

Upcoming Phases

Phase 5: Python Cross-Language Validation

Why: V2 claims Python support via python_parser.py and SOURCE_LANGUAGE_BY_SUFFIX, but all 14 evaluation tasks are TypeScript/JavaScript. Python AST parsing is fundamentally different from JS/TS regex parsing, and the retrieval pipeline should be validated on Python.

Available repos:

  • eval-repos/pypi/click/ — CLI framework (commands, options, arguments, types)
  • eval-repos/pypi/requests/ — HTTP library (sessions, adapters, auth, redirects)
  • eval-repos/pypi/rich/ — Terminal UI (console, tables, progress, layout)

Approach:

  1. Study each library's structure (2-3 hours)
  2. Identify realistic multi-goal bugs (1-2 per library)
  3. Create task definitions and goal labels
  4. Run 3×3 repeated-run protocol

Target: 2-3 Python tasks across 2 libraries

Success Criteria:

  • At least 2/3 tasks elapsed-positive
  • Comparable tool/token efficiency to TS tasks
  • No regression in existing tasks

Effort estimate: 0.5-1 day

Phase 6: Production Readiness

Items:

  1. Clean up freeze-snapshots/ directory
  2. Add .pytest_tmp/ to .gitignore
  3. Verify CI/CD pipeline (GitHub Actions)
  4. Run pre-release check script
  5. Package smoke test
  6. Final documentation pass
  7. Tag v2.0.0-beta

Effort estimate: 0.5 day

Benchmark Coverage Matrix

By Domain

Domain Tasks Status
Middleware/Auth 2 ✅ V2 baseline
Route/Guard 1 ✅ V2 baseline
Security/Config 1 ✅ Phase 3
Validation/Schema 1 ✅ Phase 3
Frontend Auth 1 ✅ Phase 3
ORM/Database 1 ✅ Phase 3
Framework Config 1 ✅ Phase 3
File Handling 1 🔄 Phase 4
Email/Notifications 1 🔄 Phase 4
Data Serialization 1 🔄 Phase 4
CLI/Commands 0 ⏳ Phase 5 (Python)
HTTP Client 0 ⏳ Phase 5 (Python)
Terminal UI 0 ⏳ Phase 5 (Python)

By Language

Language Tasks Status
TypeScript/JavaScript 11 ✅ Complete
Python 0 ⏳ Phase 5

By Framework

Framework Tasks
NestJS 6
Express 2
AdonisJS 2
React/Next.js 1
Click (Python) 0
Requests (Python) 0

Test Health

Metric Value
Total tests 487
Passing 484
Pre-existing failures 3 (test_self_evaluation.py)
New tests since V2 start +192 (+65%)

Git History

9490225 feat(phase4): add file upload, mail i18n, and serializer interceptor tasks
f61dc3c merge: V2 multi-goal retrieval baseline (main)
5e61f29 fix(v2): add security and pipe keyword sets
9de45bd feat(phase3): add benchmark expansion tasks and evaluation infrastructure
ac67177 feat(v2): freeze multi-goal retrieval baseline with quality hardening
v2.0.0-alpha  ← tag

Execution Order

Now:      Phase 4 evaluation (3 tasks × 3 repeats)
          Update V2_PROGRESS doc
          
Next:     Phase 5 Python tasks (research + create + evaluate)
          
Then:     Phase 6 Production readiness
          Tag v2.0.0-beta
          
Future:   Optional LLM planner (when rule expansion returns diminish)
          Optional embedding clustering (when path clustering fails)