Skip to content

Conversation

@richardwooding
Copy link
Contributor

Summary

Implements comprehensive support for PostgreSQL multi-dimensional arrays with automatic dimension detection from database schema metadata.

Fixes #49

Changes

Core Implementation

  1. Schema Enhancement (pg/provider.go)

    • Added Dimensions int field to FieldSchema struct
    • Implemented detectArrayDimensions() function to parse PostgreSQL type strings
    • Updated LoadTableSchema() to query udt_name column and detect dimensions
    • Handles both bracket notation (integer[][]) and underscore notation (_int4[])
    • Correctly combines notations (e.g., _int4[] = 2D array)
  2. SQL Generation (cel2sql.go)

    • Updated getArrayDimension() to use CEL typeMap for accurate type lookup
    • Modified ARRAY_LENGTH generation to use detected dimensions: ARRAY_LENGTH(field, dimension)
    • Supports 1D through 4D+ arrays

Test Coverage

  1. Unit Tests (pg/dimension_detection_test.go)

    • 22 test cases for detectArrayDimensions()
    • Covers scalar types, 1D-4D arrays, both notations, edge cases
  2. Integration Tests (multidim_array_test.go)

    • Tests 1D, 2D, 3D, and 4D array size() operations
    • Backward compatibility tests (defaults to 1D without schema)
    • Explicit vs implicit dimension specification tests

Dimension Detection Logic

// Examples:
"integer"0 (not an array)
"integer[]"1 (1D array)
"_int4"1 (1D array, underscore notation)
"integer[][]"2 (2D array)
"_int4[]"2 (2D array, underscore + bracket)
"integer[][][]"3 (3D array)
"_int4[][]"3 (3D array)

Backward Compatibility

Fully backward compatible

  • Arrays without schema information default to dimension 1
  • Explicit Dimensions=0 defaults to dimension 1
  • Existing code continues to work without changes
  • All 238 existing tests pass

Example Usage

With Schema (Automatic Detection)

// Database schema introspection automatically detects dimensions
provider, err := pg.NewTypeProviderWithConnection(ctx, connString)
err = provider.LoadTableSchema(ctx, "products")

env, err := cel.NewEnv(
    cel.CustomTypeProvider(provider),
    cel.Variable("product", cel.ObjectType("products")),
)

// For a 2D array field 'matrix integer[][]':
ast, _ := env.Compile("size(product.matrix) > 0")
sql, _ := cel2sql.Convert(ast, cel2sql.WithSchemas(provider.GetSchemas()))
// Output: COALESCE(ARRAY_LENGTH(product.matrix, 2), 0) > 0

Manual Schema Definition

schema := pg.NewSchema([]pg.FieldSchema{
    {Name: "tags", Type: "text", Repeated: true, Dimensions: 1},      // 1D
    {Name: "matrix", Type: "integer", Repeated: true, Dimensions: 2},  // 2D
    {Name: "cube", Type: "float", Repeated: true, Dimensions: 3},      // 3D
})

Test Results

✓ All 238 tests pass
✓ Coverage: 72.2%
✓ No lint issues
✓ Backward compatibility verified

Related Issues

🤖 Generated with Claude Code

Implemented comprehensive support for PostgreSQL multi-dimensional arrays
with automatic dimension detection and backward compatibility.

Changes:
- Added Dimensions field to pg.FieldSchema to store array dimensionality
- Implemented detectArrayDimensions() to parse PostgreSQL type strings
  * Handles bracket notation: integer[], integer[][], integer[][][]
  * Handles underscore notation: _int4, _int4[], _int4[][]
  * Correctly combines both notations (e.g., _int4[] = 2D array)
- Updated LoadTableSchema() to detect dimensions from udt_name column
- Modified ARRAY_LENGTH SQL generation to use detected dimensions
- Updated getArrayDimension() to use CEL typeMap for accurate type lookup

Testing:
- Added comprehensive unit tests for dimension detection (22 test cases)
- Added integration tests for 1D, 2D, 3D, and 4D arrays
- Added backward compatibility tests
- All existing tests pass (72.2% coverage)

Backward Compatibility:
- Arrays without schema information default to dimension 1
- Explicit Dimensions=0 defaults to dimension 1
- Existing code continues to work without changes

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
@richardwooding richardwooding merged commit 6bc4c02 into main Oct 31, 2025
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Hardcoded Array Dimension Limits Multi-Dimensional Array Support

1 participant