Skip to content

Conversation

@pyramation
Copy link
Collaborator

@pyramation pyramation commented Jul 1, 2025

Add @pgsql/traverse package for PostgreSQL AST traversal

Summary

This PR introduces a new @pgsql/traverse package that provides PostgreSQL AST traversal utilities using a visitor pattern similar to Babel's traverse functionality. The package offers two APIs:

  1. Modern walk API: Features a NodePath class for better context tracking, early return mechanism for skipping subtrees, and runtime schema integration for precise traversal
  2. Legacy visit API: Maintains backward compatibility with a simpler visitor pattern

Key features:

  • Runtime schema integration: Uses the generated runtime schema from @pgsql/transform to precisely identify which fields are Node types that need traversal
  • Type-safe traversal: Leverages TypeScript for better type checking and IDE support
  • Flexible visitor pattern: Supports both function-based and object-based visitors
  • Early return mechanism: Allows visitors to return false to skip subtree traversal
  • Comprehensive test coverage: 10 tests covering various AST structures and edge cases

Review & Testing Checklist for Human

  • Test traverse functionality with real PostgreSQL parse results - Import actual parsed SQL and verify both walk and visit APIs correctly traverse all expected nodes
  • Verify cross-package dependencies work in monorepo - Ensure the runtime schema import from @pgsql/transform resolves correctly when published
  • Test complex AST structures - Try nested SELECT statements, JOINs, WHERE clauses to ensure deep traversal works correctly
  • Validate early return mechanism - Test that returning false from a visitor actually prevents traversal into child nodes
  • Check TypeScript compilation - Build the entire monorepo to ensure no TypeScript errors in the CI environment

Recommended test plan: Create a small script that parses a complex SQL statement and uses both APIs to traverse and collect all node types, comparing results to ensure consistency.


Diagram

%%{ init : { "theme" : "default" }}%%
graph TB
    subgraph "transform package"
        T1["packages/transform/src/index.ts"]:::minor-edit
        T2["packages/transform/src/17/runtime-schema.ts"]:::context
    end
    
    subgraph "traverse package (NEW)"
        N1["packages/traverse/package.json"]:::major-edit
        N2["packages/traverse/src/traverse.ts"]:::major-edit
        N3["packages/traverse/__test__/traverse.test.ts"]:::major-edit
        N4["packages/traverse/README.md"]:::major-edit
    end
    
    subgraph "types package"
        TY1["@pgsql/types"]:::context
    end
    
    T1 -->|exports runtime schema| N2
    T2 -->|runtime schema data| T1
    TY1 -->|Node types| N2
    N2 -->|tested by| N3
    
    subgraph Legend
        L1["Major Edit"]:::major-edit
        L2["Minor Edit"]:::minor-edit
        L3["Context/No Edit"]:::context
    end
    
    classDef major-edit fill:#90EE90
    classDef minor-edit fill:#87CEEB  
    classDef context fill:#FFFFFF
Loading

Notes

  • Session requester: Dan Lynch (@pyramation)
  • Devin session: https://app.devin.ai/sessions/f0ff151c05b04af692c23ae28661deab
  • TypeScript compilation fix: Had to resolve cross-package import issues by properly exporting runtime schema from transform package index
  • ParseResult traversal: Required special handling since ParseResult is not in the runtime schema (it's a wrapper type, not a PostgreSQL AST node)
  • Backward compatibility: The legacy visit function maintains the exact same API as the original pseudocode provided
  • Runtime schema usage: Currently uses PG17 runtime schema; may need updates for other PostgreSQL versions in the future

@devin-ai-integration
Copy link
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

- Implement visitor pattern for PostgreSQL AST nodes
- Add comprehensive test suite with various node types
- Follow existing package structure and build patterns
- Support both individual nodes and ParseResult objects
- Include Jest configuration for TypeScript testing
- Add README with usage examples and API documentation

Co-Authored-By: Dan Lynch <[email protected]>
- Add NodePath class for better context handling with proper TypeScript generics
- Implement early return mechanism (return false to skip subtrees)
- Use runtime schema for precise traversal instead of guessing which fields are nodes
- Add comprehensive tests for new functionality including NodePath properties and early return
- Maintain backward compatibility with existing visit API as wrapper around new walk API
- Update documentation with new walk API examples and NodePath class details
- Fix traversal logic to handle both schema-guided and fallback traversal modes

Co-Authored-By: Dan Lynch <[email protected]>
…mports

- Export runtime schema from @pgsql/transform package index
- Add @pgsql/transform as dependency to @pgsql/traverse
- Use proper package import instead of direct file import
- Maintain all enhanced traverse functionality (NodePath, early return, runtime schema)

Co-Authored-By: Dan Lynch <[email protected]>
- Fix ParseResult nodes test by implementing proper recursive traversal in visit function
- Correct context path generation to exclude node type names
- All 10 tests now pass locally
- Maintain backward compatibility with legacy visit API

Co-Authored-By: Dan Lynch <[email protected]>
@pyramation pyramation merged commit 9cc3436 into main Jul 1, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants