Skip to content

Commit 3c7b9ba

Browse files
feat: Add comprehensive performance benchmarks to CI/CD (fixes #52)
Add complete benchmark suite covering all major conversion features: **Benchmark Coverage:** - Simple operations: field comparisons, boolean checks - Operators: logical, arithmetic, string operations - Comprehensions: all, exists, exists_one, filter, map - JSON/JSONB: nested field access, has() checks - Regex: pattern matching with RE2 to POSIX conversion - Complex expressions: deeply nested, large expressions - Timestamps: date/time operations and comparisons - String operations: startsWith, endsWith, contains - Query analysis: index recommendation performance - Conversion options: schema, depth, output length limits **CI/CD Integration:** - Benchmark job runs automatically on PR and main push - Stores benchmark results for historical tracking - Alerts on performance degradation >150% - GitHub Actions integration with benchmark-action **Makefile Targets:** - make bench: Run all benchmarks - make bench-compare: Save benchmarks for comparison **Documentation:** - Comprehensive benchmark guide in CLAUDE.md - Examples of running and comparing benchmarks - Benchmark output interpretation guide All benchmarks passing with zero panics or failures. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
1 parent fadc687 commit 3c7b9ba

File tree

4 files changed

+662
-1
lines changed

4 files changed

+662
-1
lines changed

.github/workflows/ci.yml

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,3 +46,35 @@ jobs:
4646
file: ./coverage.out
4747
fail_ci_if_error: false
4848
token: ${{ secrets.CODECOV_TOKEN }} # Add token for better reliability
49+
50+
benchmark:
51+
name: Benchmark
52+
runs-on: ubuntu-latest
53+
steps:
54+
- name: Checkout code
55+
uses: actions/checkout@v4
56+
57+
- name: Set up Go
58+
uses: actions/setup-go@v5
59+
with:
60+
go-version: '1.24.x'
61+
cache: true
62+
check-latest: true
63+
64+
- name: Download dependencies
65+
run: go mod download
66+
67+
- name: Run benchmarks
68+
run: go test -bench=. -benchmem -benchtime=1s -run=^$$ ./... | tee benchmark-results.txt
69+
70+
- name: Store benchmark result
71+
uses: benchmark-action/github-action-benchmark@v1
72+
with:
73+
tool: 'go'
74+
output-file-path: benchmark-results.txt
75+
github-token: ${{ secrets.GITHUB_TOKEN }}
76+
auto-push: false
77+
comment-on-alert: true
78+
alert-threshold: '150%'
79+
fail-on-alert: false
80+
summary-always: true

CLAUDE.md

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,8 @@ cel2sql converts CEL (Common Expression Language) expressions to PostgreSQL SQL
1717
make build # Build the project
1818
make test # Run tests with race detection and coverage
1919
make test-coverage # Generate HTML coverage report
20+
make bench # Run performance benchmarks
21+
make bench-compare # Run benchmarks and save for comparison
2022
```
2123

2224
### Code Quality
@@ -167,6 +169,133 @@ These validations prevent PostgreSQL syntax errors and ensure predictable behavi
167169
- Verify SQL output matches PostgreSQL syntax (single quotes, proper functions)
168170
- Use testcontainers for integration tests with real PostgreSQL
169171

172+
### Performance Benchmarks
173+
174+
cel2sql includes comprehensive benchmarks to track performance and detect regressions. Benchmarks are automatically run as part of the CI/CD pipeline.
175+
176+
#### Running Benchmarks Locally
177+
178+
```bash
179+
# Run all benchmarks
180+
make bench
181+
182+
# Run benchmarks and save output for comparison
183+
make bench-compare
184+
185+
# Run specific benchmark
186+
go test -bench=BenchmarkConvertSimple -benchmem ./...
187+
188+
# Run benchmarks with custom duration
189+
go test -bench=. -benchmem -benchtime=5s ./...
190+
```
191+
192+
#### Benchmark Categories
193+
194+
The benchmark suite covers all major conversion features:
195+
196+
**Simple Operations** (`BenchmarkConvertSimple`)
197+
- Field comparisons (equality, greater than, less than)
198+
- String operations (equality checks)
199+
- Boolean checks
200+
201+
**Operators** (`BenchmarkConvertOperators`)
202+
- Logical operators (AND, OR)
203+
- Arithmetic operators (+, -, *, /, %)
204+
- String concatenation
205+
- Complex nested expressions
206+
207+
**Comprehensions** (`BenchmarkConvertComprehensions`)
208+
- `all()` - Universal quantification
209+
- `exists()` - Existential quantification
210+
- `exists_one()` - Unique existence
211+
- `filter()` - Array filtering
212+
- `map()` - Array transformation
213+
214+
**JSON/JSONB** (`BenchmarkConvertJSONPath`)
215+
- Simple and nested JSON field access
216+
- JSON field existence checks (`has()`)
217+
- JSON field comparisons
218+
- Complex JSON expressions
219+
220+
**Regex** (`BenchmarkConvertRegex`)
221+
- Simple patterns
222+
- Case-insensitive patterns
223+
- Character classes (\\d, \\w, \\s)
224+
- Word boundaries (\\b)
225+
226+
**Complex Expressions** (`BenchmarkConvertDeeplyNested`, `BenchmarkConvertLargeExpression`)
227+
- Deeply nested AND/OR chains
228+
- Nested parentheses
229+
- Ternary operators
230+
- Large expressions with many conditions
231+
232+
**Timestamps** (`BenchmarkConvertTimestamps`)
233+
- Timestamp comparisons
234+
- Date/time function calls
235+
- DateTime operations
236+
237+
**String Operations** (`BenchmarkConvertStringOperations`)
238+
- startsWith, endsWith, contains
239+
- String concatenation
240+
- Multiple string operations combined
241+
242+
**Query Analysis** (`BenchmarkAnalyzeQuery`)
243+
- Index recommendation generation
244+
- Pattern detection for optimization
245+
246+
**Options** (`BenchmarkConvertWithOptions`)
247+
- Various conversion option combinations
248+
- Schema usage overhead
249+
- Max depth and output length limits
250+
251+
#### Benchmark Output
252+
253+
Benchmarks report:
254+
- **Iterations**: Number of times the operation was executed
255+
- **ns/op**: Nanoseconds per operation (lower is better)
256+
- **B/op**: Bytes allocated per operation (lower is better)
257+
- **allocs/op**: Number of allocations per operation (lower is better)
258+
259+
Example output:
260+
```
261+
BenchmarkConvertSimple/equality-12 1412060 848.7 ns/op 1593 B/op 25 allocs/op
262+
BenchmarkConvertOperators/logical_and-12 943438 1255 ns/op 2154 B/op 36 allocs/op
263+
```
264+
265+
#### CI/CD Integration
266+
267+
Benchmarks run automatically on every PR and push to main:
268+
- Runs on Go 1.24.x
269+
- Stores benchmark results for historical tracking
270+
- Comments on PRs if performance degrades >150%
271+
- Provides summary of all benchmark results
272+
273+
#### Comparing Benchmark Results
274+
275+
To compare benchmarks between two runs:
276+
277+
```bash
278+
# Run benchmarks and save baseline
279+
make bench-compare # Saves to bench-new.txt
280+
281+
# Make changes to code
282+
283+
# Run benchmarks again and compare
284+
mv bench-new.txt bench-old.txt
285+
make bench-compare
286+
287+
# Install benchstat for detailed comparison (optional)
288+
go install golang.org/x/perf/cmd/benchstat@latest
289+
benchstat bench-old.txt bench-new.txt
290+
```
291+
292+
#### When to Run Benchmarks
293+
294+
- Before and after performance optimizations
295+
- When modifying core conversion logic
296+
- When adding new features that may impact performance
297+
- To validate that changes don't cause regressions
298+
170299
## Common Patterns
171300

172301
### Creating Type Providers

Makefile

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Makefile for cel2sql project
22

3-
.PHONY: build test lint fmt clean help install-tools deps vuln-check
3+
.PHONY: build test bench lint fmt clean help install-tools deps vuln-check
44

55
# Build the project
66
build:
@@ -10,6 +10,17 @@ build:
1010
test:
1111
go test -v -race -coverprofile=coverage.out -covermode=atomic ./...
1212

13+
# Run benchmarks
14+
bench:
15+
go test -bench=. -benchmem -run=^$$ ./...
16+
17+
# Run benchmarks with comparison output
18+
bench-compare:
19+
@echo "Running benchmarks and saving to bench-new.txt"
20+
@go test -bench=. -benchmem -run=^$$ ./... | tee bench-new.txt
21+
@echo ""
22+
@echo "Compare with previous run using: benchstat bench-old.txt bench-new.txt"
23+
1324
# Run tests with coverage report
1425
test-coverage: test
1526
go tool cover -html=coverage.out -o coverage.html
@@ -65,6 +76,8 @@ help:
6576
@echo "Available targets:"
6677
@echo " build - Build the project"
6778
@echo " test - Run tests"
79+
@echo " bench - Run benchmarks"
80+
@echo " bench-compare - Run benchmarks and save for comparison"
6881
@echo " test-coverage - Run tests with coverage report"
6982
@echo " lint - Run linting"
7083
@echo " fmt - Format code"

0 commit comments

Comments
 (0)