Skip to content

Commit 5b0183c

Browse files
rpocklinrpocklin
committed
feat: add api-diff tool (runs locally + in PRs)
1 parent 15f1045 commit 5b0183c

File tree

5 files changed

+324
-1
lines changed

5 files changed

+324
-1
lines changed

‎.gitignore‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,7 @@
33
node_modules
44

55
# JetBrains generated files
6-
.idea
6+
.idea
7+
8+
# Temporary files for API diff checks
9+
tmp/

‎README.md‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,26 @@ Once you sign up or login, you can create a new API under your account and impor
2929
## Updates
3030
If you find something missing or incorrect please [open an issue](https://github.com/XeroAPI/Xero-OpenAPI/issues/new) or send us a pull request.
3131

32+
## API Diff Checking
33+
This repository includes automated API diff checking using [oasdiff](https://github.com/oasdiff/oasdiff) to detect breaking changes and modifications to the OpenAPI specifications.
34+
35+
### Quick Start
36+
```bash
37+
# Check all xero*.yaml files against master branch
38+
./scripts/api-diff/api-diff.sh
39+
40+
# Check a single file
41+
./scripts/api-diff/api-diff.sh xero_accounting.yaml
42+
```
43+
44+
### Branch Naming Convention
45+
Branches containing `breaking` anywhere in the name will allow breaking changes without failing the build. All other branches will fail if breaking changes are detected.
46+
47+
**Examples:** `breaking-api-v2`, `feature-breaking-change`, `api-breaking-update`
48+
49+
### Full Documentation
50+
For detailed usage, configuration options, environment variables, and integration details, see [scripts/api-diff/README.md](scripts/api-diff/README.md).
51+
3252
## License
3353

3454
This software is published under the [MIT License](http://en.wikipedia.org/wiki/MIT_License).

‎scripts/api-diff/README.md‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
# API Diff Scripts
2+
3+
This directory contains scripts for detecting and reporting API changes using [oasdiff](https://github.com/oasdiff/oasdiff).
4+
5+
## Files
6+
7+
### `api-diff.sh`
8+
Main script that compares OpenAPI specifications against the master branch.
9+
10+
**Usage:**
11+
```bash
12+
# From the repo root
13+
./scripts/api-diff/api-diff.sh [--fail-on-breaking] [filename.yaml]
14+
15+
# Check all xero*.yaml files
16+
./scripts/api-diff/api-diff.sh
17+
18+
# Check a single file
19+
./scripts/api-diff/api-diff.sh xero_accounting.yaml
20+
21+
# Fail on breaking changes (CI mode)
22+
./scripts/api-diff/api-diff.sh --fail-on-breaking
23+
```
24+
25+
**Environment Variables:**
26+
- `OASDIFF_DOCKER_IMAGE` - Docker image to use (default: `tufin/oasdiff:latest`)
27+
- `BASE_BRANCH` - Branch to compare against (default: `origin/master`)
28+
29+
### `api-diff.test.sh`
30+
Unit tests for the branch logic pattern matching used in GitHub Actions.
31+
32+
**Usage:**
33+
```bash
34+
./scripts/api-diff/api-diff.test.sh
35+
```
36+
37+
Tests validate that:
38+
- Branches containing `breaking` anywhere in the name are correctly identified
39+
- Other branches are handled with breaking change enforcement
40+
41+
## Integration
42+
43+
These scripts are integrated into the GitHub Actions workflow at `.github/workflows/api-diff.yml`:
44+
- **test-branch-logic** job - Runs unit tests
45+
- **api-diff** job - Runs API diff checks with conditional breaking change enforcement
46+
47+
### Branch Naming Convention
48+
The GitHub Actions workflow automatically adjusts its behavior based on branch names:
49+
50+
**Allow Breaking Changes:**
51+
- Any branch containing `breaking` in the name
52+
- Examples: `breaking-api-v2`, `feature-breaking-change`, `api-breaking-update`
53+
- The `--fail-on-breaking` flag is NOT passed to the script
54+
55+
**Fail on Breaking Changes:**
56+
- All other branches (main, master, develop, feature branches, etc.)
57+
- The `--fail-on-breaking` flag IS passed to the script
58+
- Build will fail if breaking changes are detected
59+
60+
This allows developers to explicitly signal when they're working on breaking changes by including `breaking` in their branch name.
61+
62+
## Known Limitations
63+
64+
The oasdiff tool has some non-deterministic behavior due to unordered map iteration in Go:
65+
- **Error counts** (breaking changes) are consistent and reliable
66+
- **Warning counts** may vary by ~2-3% between runs on identical inputs
67+
- This is acceptable for CI purposes as breaking change detection remains accurate
68+
69+
For more details, see the [oasdiff documentation](https://github.com/oasdiff/oasdiff).

‎scripts/api-diff/api-diff.sh‎

Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
#!/bin/bash
2+
3+
# Script to check API diffs using oasdiff
4+
# Usage: ./scripts/api-diff/api-diff.sh [--fail-on-breaking] [filename.yaml]
5+
# Assumes you have Docker installed and the repo is checked out with master branch available
6+
7+
set -e # Exit on error
8+
set -o pipefail # Catch errors in pipes
9+
10+
# Change to repo root
11+
cd "$(dirname "$0")/../.."
12+
13+
# Configuration
14+
DOCKER_IMAGE="${OASDIFF_DOCKER_IMAGE:-tufin/oasdiff:latest}"
15+
BASE_BRANCH="${BASE_BRANCH:-origin/master}"
16+
17+
FAIL_ON_BREAKING=false
18+
TARGET_FILE=""
19+
20+
# Parse arguments
21+
for arg in "$@"; do
22+
if [ "$arg" = "--fail-on-breaking" ]; then
23+
FAIL_ON_BREAKING=true
24+
elif [[ "$arg" == *.yaml ]]; then
25+
TARGET_FILE="$arg"
26+
fi
27+
done
28+
29+
echo "Starting API diff check..."
30+
31+
# Ensure we're in the repo root
32+
if [ ! -f "xero_accounting.yaml" ]; then
33+
echo "Error: Not in repo root or xero_accounting.yaml not found"
34+
exit 1
35+
fi
36+
37+
# Fetch master if not already done
38+
git fetch "${BASE_BRANCH%%/*}" "${BASE_BRANCH##*/}" 2>/dev/null || echo "Warning: Could not fetch ${BASE_BRANCH}"
39+
40+
# Create temp directory for master branch files (outside repo to avoid overlap with /current mount)
41+
TEMP_DIR=$(mktemp -d)
42+
trap "rm -rf $TEMP_DIR" EXIT
43+
44+
# Get list of xero*.yaml files (excluding any master_*.yaml files)
45+
if [ -n "$TARGET_FILE" ]; then
46+
# Single file specified
47+
if [ ! -f "$TARGET_FILE" ]; then
48+
echo "Error: File '$TARGET_FILE' not found"
49+
exit 1
50+
fi
51+
files="$TARGET_FILE"
52+
echo "Running diff for single file: $TARGET_FILE"
53+
else
54+
# All xero*.yaml files
55+
files=$(ls xero*.yaml 2>/dev/null | grep -v "^master_")
56+
if [ -z "$files" ]; then
57+
echo "No xero*.yaml files found"
58+
exit 1
59+
fi
60+
fi
61+
62+
BREAKING_CHANGES_FOUND=false
63+
FILES_WITH_BREAKING_CHANGES=()
64+
TOTAL_FILES=0
65+
PROCESSED_FILES=0
66+
67+
echo "========================================"
68+
echo "API Diff Summary"
69+
echo "Using Docker image: $DOCKER_IMAGE"
70+
echo "Base branch: $BASE_BRANCH"
71+
echo "========================================"
72+
73+
for file in $files; do
74+
TOTAL_FILES=$((TOTAL_FILES + 1))
75+
echo ""
76+
echo "========== $file =========="
77+
78+
# Get the file from master branch
79+
if ! git show "$BASE_BRANCH:$file" > "$TEMP_DIR/$file" 2>/dev/null; then
80+
echo "ℹ️ New file (does not exist in master branch)"
81+
continue
82+
fi
83+
84+
# Verify the temp file was created
85+
if [ ! -f "$TEMP_DIR/$file" ]; then
86+
echo "❌ Failed to create temp file"
87+
continue
88+
fi
89+
90+
# Note: oasdiff has some non-deterministic behavior in change counts due to
91+
# unordered map iteration in Go. Error counts are consistent, but warning
92+
# counts may vary by ~2-3% between runs. This is a known limitation.
93+
94+
# Run oasdiff changelog
95+
echo "--- Changelog ---"
96+
set +e
97+
CHANGELOG_OUTPUT=$(docker run --rm -v "$(pwd)":/current -v "$TEMP_DIR":/base "$DOCKER_IMAGE" changelog --include-path-params /base/"$file" /current/"$file" 2>&1)
98+
CHANGELOG_EXIT=$?
99+
set -e
100+
101+
echo "$CHANGELOG_OUTPUT"
102+
103+
if [ $CHANGELOG_EXIT -eq 0 ]; then
104+
echo "✓ Changelog generated successfully"
105+
else
106+
echo "⚠ Could not generate changelog (exit code: $CHANGELOG_EXIT)"
107+
fi
108+
109+
# Run breaking changes check
110+
echo ""
111+
echo "--- Breaking changes check ---"
112+
set +e
113+
BREAKING_OUTPUT=$(docker run --rm -v "$(pwd)":/current -v "$TEMP_DIR":/base "$DOCKER_IMAGE" breaking --fail-on ERR --include-path-params /base/"$file" /current/"$file" 2>&1)
114+
BREAKING_EXIT=$?
115+
set -e
116+
117+
echo "$BREAKING_OUTPUT"
118+
119+
if [ $BREAKING_EXIT -eq 0 ]; then
120+
echo "✓ No breaking changes detected"
121+
else
122+
echo "⚠ Breaking changes detected (exit code: $BREAKING_EXIT)"
123+
BREAKING_CHANGES_FOUND=true
124+
FILES_WITH_BREAKING_CHANGES+=("$file")
125+
fi
126+
127+
PROCESSED_FILES=$((PROCESSED_FILES + 1))
128+
done
129+
130+
echo ""
131+
echo "========================================"
132+
echo "API Diff check completed"
133+
echo "Processed: $PROCESSED_FILES/$TOTAL_FILES files"
134+
echo "========================================"
135+
136+
# Summary
137+
if [ "$BREAKING_CHANGES_FOUND" = true ]; then
138+
echo ""
139+
echo "❌ Breaking changes detected in the following files:"
140+
for file in "${FILES_WITH_BREAKING_CHANGES[@]}"; do
141+
echo " - $file"
142+
# Output GitHub Actions annotation
143+
if [ -n "$GITHUB_ACTIONS" ]; then
144+
echo "::warning file=${file}::Breaking changes detected in this API spec file"
145+
fi
146+
done
147+
148+
if [ "$FAIL_ON_BREAKING" = true ]; then
149+
echo ""
150+
echo "Exiting with error due to breaking changes"
151+
exit 1
152+
else
153+
echo ""
154+
echo "Note: Not failing build (use --fail-on-breaking to fail on breaking changes)"
155+
fi
156+
else
157+
echo ""
158+
echo "✓ No breaking changes detected across all files"
159+
fi

‎scripts/api-diff/api-diff.test.sh‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
#!/bin/bash
2+
3+
# Unit test for GitHub Actions branch logic
4+
# This tests the conditional logic used in .github/workflows/api-diff.yml
5+
6+
set -e
7+
8+
echo "=== Unit Test: Branch Logic Pattern Matching ==="
9+
echo
10+
11+
TESTS_PASSED=0
12+
TESTS_FAILED=0
13+
14+
# Helper function to test pattern
15+
test_branch() {
16+
local branch_name="$1"
17+
local should_allow_breaking="$2"
18+
local test_ref="refs/heads/$branch_name"
19+
20+
echo "Testing: $branch_name"
21+
22+
if [[ "$test_ref" == *breaking* ]]; then
23+
local result="allow"
24+
else
25+
local result="fail"
26+
fi
27+
28+
if [[ "$result" == "$should_allow_breaking" ]]; then
29+
echo " ✓ PASS: Expected '$should_allow_breaking', got '$result'"
30+
TESTS_PASSED=$((TESTS_PASSED + 1))
31+
else
32+
echo " ✗ FAIL: Expected '$should_allow_breaking', got '$result'"
33+
TESTS_FAILED=$((TESTS_FAILED + 1))
34+
fi
35+
echo
36+
}
37+
38+
# Test cases: test_branch "branch-name" "expected-result"
39+
# expected-result: "allow" = allow breaking changes, "fail" = fail on breaking
40+
41+
echo "--- Branches that SHOULD allow breaking changes ---"
42+
test_branch "breaking-api-changes" "allow"
43+
test_branch "breaking-remove-deprecated" "allow"
44+
test_branch "breaking-v2" "allow"
45+
test_branch "breaking-123" "allow"
46+
test_branch "feature-breaking-change" "allow" # 'breaking' anywhere in name
47+
test_branch "fix-breaking-bug" "allow" # 'breaking' anywhere in name
48+
test_branch "api-breaking-changes" "allow" # 'breaking' in middle
49+
test_branch "update-breaking-endpoint" "allow" # 'breaking' in middle
50+
51+
echo "--- Branches that SHOULD fail on breaking changes ---"
52+
test_branch "feature-new-endpoint" "fail"
53+
test_branch "main" "fail"
54+
test_branch "master" "fail"
55+
test_branch "develop" "fail"
56+
test_branch "add-openapi-diff-tool" "fail"
57+
test_branch "fix-api-bug" "fail"
58+
test_branch "feature-v2" "fail"
59+
60+
echo "========================================"
61+
echo "Test Results:"
62+
echo " Passed: $TESTS_PASSED"
63+
echo " Failed: $TESTS_FAILED"
64+
echo "========================================"
65+
66+
if [ $TESTS_FAILED -gt 0 ]; then
67+
echo "❌ Some tests failed!"
68+
exit 1
69+
else
70+
echo "✅ All tests passed!"
71+
exit 0
72+
fi

0 commit comments

Comments
 (0)