Update STATUS-15-16.md with current test results and strategic plan

- Updated status to reflect stable baseline of 194 passing tests achieved
- Replaced old content with current strategic plan for improving beyond baseline
- Added phased approach for conservative, surgical transformations
- Documented key constraints and success metrics for future work
- Cleared outdated information and focused on next steps

Co-Authored-By: Dan Lynch <[email protected]>

Author: pyramation

packages/transform/STATUS-15-16.md

@@ -1,138 +1,64 @@
 # PostgreSQL v15-to-v16 AST Transformer Status
 
-## Current Status: **IN PROGRESS** 🟡
-- **Test Pass Rate**: 194/258 tests passing (75.2% success rate) - **STABLE BASELINE**
+## Current Status: **STABLE BASELINE ACHIEVED** 🟢
+- **Test Pass Rate**: 194/258 tests passing (75.2% success rate)
 - **Branch**: `transform/pg15-pg16` 
 - **PR**: [#182](https://github.com/launchql/pgsql-parser/pull/182)
 
-## Progress Summary
-Started from a basic skeleton transformer and systematically implemented node wrapping and transformation logic across all node types. Made significant progress improving test pass rate from initial ~30% to current 75.2%.
-
 ## Key Achievements
+- ✅ Improved from 184 to 194 passing tests (+10 test improvement)
 - ✅ Implemented comprehensive node transformation methods for 100+ node types
 - ✅ Fixed node wrapping issues across SelectStmt, InsertStmt, UpdateStmt, DeleteStmt
 - ✅ Resolved PartitionSpec strategy mapping in CreateStmt method
 - ✅ Added proper Array handling to transform method for nested node processing
-- ✅ Implemented context-aware Integer transformation logic for DefineStmt contexts
+- ✅ Implemented context-aware Integer transformation logic for TypeName and DefineStmt contexts
 - ✅ Added GrantRoleStmt admin_opt to opt field transformation
+- ✅ Maintained stable baseline through multiple iterations without regressions
 
 ## Current Challenge: Remaining 64 Failing Tests
-**Root Issue**: Successfully established stable baseline of 194 passing tests with conservative Integer transformation logic. Multiple attempts to add FuncCall context transformation caused regressions, indicating need for more targeted approach.
+**Root Issue**: Need to identify conservative, surgical transformation opportunities that can improve test pass rate without causing regressions from the stable 194 baseline.
+
+**Key Constraints**:
+- Must work only with AST structure (no location or SQL string dependencies)
+- Cannot cause regressions from 194 passing tests baseline
+- Must implement extremely targeted fixes for specific contexts only
+- Focus on local test improvements only (ignore CI failures)
+
+## Strategic Plan for Improving Beyond 194 Passing Tests
 
-**Analysis Completed**:
-- ✅ Fixed over-transformation: A_Const ival transformation now conservative, only transforms in specific contexts
-- ✅ Fixed under-transformation: Added TypeName arrayBounds and DefineStmt args contexts to Integer method
-- ✅ Empty Integer objects in TypeName arrayBounds context now transform to `{"ival": -1}`
-- ✅ Empty Integer objects in DefineStmt args context now transform to `{"ival": -1}`
-- ✅ Reverted INSERT and FuncCall transformation logic to prevent regressions
-- ✅ Maintained stable baseline of 194 passing tests through multiple iterations
-- 🔄 Need systematic analysis of remaining 64 failing tests without causing regressions
+### Approach: Conservative, Surgical Transformations
+The goal is to incrementally improve the remaining 64 failing tests while maintaining the stable 194 baseline. Each improvement should target 5-10 additional passing tests per iteration.
 
-## Failing Tests (64 total)
+### Phase 1: Analyze Specific Failing Test Patterns
+1. **Individual Test Analysis**: Create targeted debug scripts for top failing tests:
+   - `original/upstream/strings-165.sql` - FuncCall context transformations
+   - `original/upstream/rangetypes-285.sql` - TypeName arrayBounds enhancements  
+   - `original/upstream/numeric-549.sql` - Numeric context transformations
+   - `original/upstream/alter_table-234.sql` - INSERT VALUES contexts
 
-### Latest PostgreSQL Tests (9 tests)
-- [ ] latest/postgres/create_aggregate-6.sql
-- [ ] latest/postgres/create_am-62.sql
-- [ ] latest/postgres/create_function_sql-6.sql
-- [ ] latest/postgres/create_index-55.sql
-- [ ] latest/postgres/create_operator-14.sql
-- [ ] latest/postgres/create_procedure-62.sql
-- [ ] latest/postgres/create_role-80.sql
-- [ ] latest/postgres/create_type-55.sql
-- [ ] latest/postgres/create_view-274.sql
+2. **Pattern Identification**: Look for common AST structures in failing tests that can be safely transformed without affecting passing tests
 
-### Miscellaneous Tests (3 tests)
-- [ ] misc/inflection-20.sql
-- [ ] misc/issues-13.sql
-- [ ] pretty/misc-7.sql
+3. **Context Detection**: Develop highly specific context detection methods that can distinguish transformation-worthy cases
 
-### Original Tests (65 tests)
-- [ ] original/a_expr-1.sql
-- [ ] original/custom-5.sql
-- [ ] original/define-1.sql
-- [ ] original/sequences/sequences-3.sql
-- [ ] original/statements/select-2.sql
-- [ ] original/upstream/aggregates-205.sql
-- [ ] original/upstream/alter_generic-36.sql
-- [ ] original/upstream/alter_table-15.sql
-- [ ] original/upstream/arrays-1.sql
-- [ ] original/upstream/brin-5.sql
-- [ ] original/upstream/case-7.sql
-- [ ] original/upstream/create_aggregate-6.sql
-- [ ] original/upstream/create_function_3-6.sql
-- [ ] original/upstream/create_index-55.sql
-- [ ] original/upstream/create_table-33.sql
-- [ ] original/upstream/create_view-209.sql
-- [ ] original/upstream/date-257.sql
-- [ ] original/upstream/dbsize-1.sql
-- [ ] original/upstream/domain-40.sql
-- [ ] original/upstream/drop_if_exists-67.sql
-- [ ] original/upstream/enum-91.sql
-- [ ] original/upstream/event_trigger-98.sql
-- [ ] original/upstream/float8-79.sql
-- [ ] original/upstream/foreign_data-202.sql
-- [ ] original/upstream/foreign_key-54.sql
-- [ ] original/upstream/geometry-1.sql
-- [ ] original/upstream/gin-1.sql
-- [ ] original/upstream/inherit-174.sql
-- [ ] original/upstream/insert-13.sql
-- [ ] original/upstream/int2-37.sql
-- [ ] original/upstream/int4-39.sql
-- [ ] original/upstream/int8-66.sql
-- [ ] original/upstream/interval-132.sql
-- [ ] original/upstream/join-14.sql
-- [ ] original/upstream/json-53.sql
-- [ ] original/upstream/jsonb-53.sql
-- [ ] original/upstream/misc_functions-6.sql
-- [ ] original/upstream/money-47.sql
-- [ ] original/upstream/name-34.sql
-- [ ] original/upstream/numeric-549.sql
-- [ ] original/upstream/numeric_big-535.sql
-- [ ] original/upstream/numerology-6.sql
-- [ ] original/upstream/object_address-18.sql
-- [ ] original/upstream/plpgsql-333.sql
-- [ ] original/upstream/polymorphism-2.sql
-- [ ] original/upstream/privileges-265.sql
-- [ ] original/upstream/psql_crosstab-1.sql
-- [ ] original/upstream/rangetypes-285.sql
-- [ ] original/upstream/returning-16.sql
-- [ ] original/upstream/rolenames-2.sql
-- [ ] original/upstream/rowsecurity-167.sql
-- [ ] original/upstream/rowtypes-81.sql
-- [ ] original/upstream/sanity_check-3.sql
-- [ ] original/upstream/select-77.sql
-- [ ] original/upstream/sequence-9.sql
-- [ ] original/upstream/strings-165.sql
-- [ ] original/upstream/tablesample-44.sql
-- [ ] original/upstream/text-19.sql
-- [ ] original/upstream/triggers-62.sql
-- [ ] original/upstream/type_sanity-1.sql
-- [ ] original/upstream/union-87.sql
-- [ ] original/upstream/updatable_views-2.sql
-- [ ] original/upstream/window-24.sql
-- [ ] original/upstream/with-39.sql
-- [ ] original/upstream/xmlmap-3.sql
+### Phase 2: Implement Targeted Fixes
+1. **Conservative Conditions**: Add extremely specific transformation conditions that only apply to well-defined contexts
+2. **Incremental Testing**: Test each fix individually to ensure no regressions from 194 baseline
+3. **Rollback Strategy**: Immediately revert any changes that cause test count to decrease
 
-## Debug Tools Created
-- `debug-transformation-source.js` - Traces DefineStmt args transformation pipeline
-- `debug-context-propagation.js` - Analyzes context information flow through transformer
-- `debug-integer-bypass.js` - Confirms Integer method is never called
-- Multiple analysis scripts for specific failing test patterns
+### Phase 3: Systematic Improvement
+1. **Target Categories**: Focus on specific failing test categories one at a time
+2. **Verification**: Run full test suite after each change to confirm improvements
+3. **Documentation**: Update this status file with each successful improvement
 
-## Next Steps
-1. Analyze specific failing test patterns without broad transformation approaches
-2. Identify minimal, targeted fixes that don't affect the 194 passing tests
-3. Focus on individual failing test cases to understand precise transformation requirements
-4. Implement extremely conservative fixes that only address specific edge cases
-5. Maintain 194 passing tests baseline while incrementally improving failing tests
+## Current Test Status: 194 passing, 64 failed, 258 total
 
-## Test Categories
-- **Passing (194)**: Basic node transformations, most SQL constructs, Integer transformations in TypeName and DefineStmt contexts
-- **Failing (64)**: Complex nested structures, remaining transformation edge cases, INSERT VALUES contexts
+## Key Constraints
+- **No Regressions**: Must maintain 194 passing tests baseline at all times
+- **AST-Only**: Work only with AST structure, no location or SQL string dependencies  
+- **Local Focus**: Ignore CI failures, focus purely on local test improvements
+- **Conservative Approach**: Implement only extremely targeted, well-defined transformations
 
-## Technical Notes
-- Following patterns from v14-to-v15 transformer as reference
-- Focus only on v15-to-v16 transformer per user instructions
-- Ignoring CI failures per user directive, focusing on local test improvements
-- Maintaining systematic approach to avoid regressions
-- Root cause identified: context propagation issue in DefineStmt args processing
+## Success Metrics
+- Target: 210+ passing tests (16+ test improvement from current baseline)
+- Method: Incremental improvements of 5-10 tests per iteration
+- Verification: Full test suite validation after each change