Skip to content

Commit a5acbc3

Browse files
the-chris-mitchellthe-chris-mitchell
authored
Merge pull request #758 from XeroAPI/add-openapi-diff-tool
Add api-diff tool (runs locally + in PRs)
2 parents b5ca731 + 02b332b commit a5acbc3

File tree

6 files changed

+416
-1
lines changed

6 files changed

+416
-1
lines changed

‎.github/workflows/api-diff.yml‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
name: OpenAPI Spec Diff Check
2+
3+
on:
4+
pull_request:
5+
push:
6+
branches:
7+
- master
8+
- main
9+
10+
concurrency:
11+
group: ${{ github.workflow }}-${{ github.ref }}
12+
cancel-in-progress: true
13+
14+
jobs:
15+
test-branch-logic:
16+
runs-on: ubuntu-latest
17+
permissions:
18+
contents: read
19+
steps:
20+
- name: Checkout code
21+
uses: actions/checkout@v4
22+
23+
- name: Run branch logic unit tests
24+
run: ./scripts/api-diff/api-diff.test.sh
25+
26+
api-diff:
27+
runs-on: ubuntu-latest
28+
needs: test-branch-logic
29+
permissions:
30+
contents: read
31+
pull-requests: write
32+
33+
steps:
34+
- name: Checkout code
35+
uses: actions/checkout@v4
36+
with:
37+
fetch-depth: 0
38+
39+
- name: Make script executable
40+
run: chmod +x scripts/api-diff/api-diff.sh
41+
42+
- name: Run API diff check
43+
run: ./scripts/api-diff/api-diff.sh

‎.gitignore‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,7 @@
33
node_modules
44

55
# JetBrains generated files
6-
.idea
6+
.idea
7+
8+
# Temporary files for API diff checks
9+
tmp/

‎README.md‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,26 @@ Once you sign up or login, you can create a new API under your account and impor
2929
## Updates
3030
If you find something missing or incorrect please [open an issue](https://github.com/XeroAPI/Xero-OpenAPI/issues/new) or send us a pull request.
3131

32+
## API Diff Checking
33+
This repository includes automated API diff checking using [oasdiff](https://github.com/oasdiff/oasdiff) to detect breaking changes and modifications to the OpenAPI specifications.
34+
35+
### Quick Start
36+
```bash
37+
# Check all xero*.yaml files against master branch
38+
./scripts/api-diff/api-diff.sh
39+
40+
# Check a single file
41+
./scripts/api-diff/api-diff.sh xero_accounting.yaml
42+
```
43+
44+
### Branch Naming Convention
45+
Branches containing `breaking` anywhere in the name will allow breaking changes without failing the build. All other branches will fail if breaking changes are detected.
46+
47+
**Examples:** `breaking-api-v2`, `feature-breaking-change`, `api-breaking-update`
48+
49+
### Full Documentation
50+
For detailed usage, configuration options, environment variables, and integration details, see [scripts/api-diff/README.md](scripts/api-diff/README.md).
51+
3252
## License
3353

3454
This software is published under the [MIT License](http://en.wikipedia.org/wiki/MIT_License).

‎scripts/api-diff/README.md‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
# API Diff Scripts
2+
3+
This directory contains scripts for detecting and reporting API changes using [oasdiff](https://github.com/oasdiff/oasdiff).
4+
5+
## Files
6+
7+
### `api-diff.sh`
8+
Main script that compares OpenAPI specifications against the master branch.
9+
10+
**Usage:**
11+
```bash
12+
# From the repo root
13+
./scripts/api-diff/api-diff.sh [--fail-on-breaking] [filename.yaml]
14+
15+
# Check all xero*.yaml files
16+
./scripts/api-diff/api-diff.sh
17+
18+
# Check a single file
19+
./scripts/api-diff/api-diff.sh xero_accounting.yaml
20+
21+
# Fail on breaking changes (CI mode)
22+
./scripts/api-diff/api-diff.sh --fail-on-breaking
23+
```
24+
25+
**Environment Variables:**
26+
- `OASDIFF_DOCKER_IMAGE` - Docker image to use (default: `tufin/oasdiff:latest`)
27+
- `BASE_BRANCH` - Branch to compare against (default: `origin/master`)
28+
29+
### `api-diff.test.sh`
30+
Unit tests for the branch logic pattern matching used in GitHub Actions.
31+
32+
**Usage:**
33+
```bash
34+
./scripts/api-diff/api-diff.test.sh
35+
```
36+
37+
Tests validate that:
38+
- Branches containing `breaking` anywhere in the name are correctly identified
39+
- Other branches are handled with breaking change enforcement
40+
41+
## Integration
42+
43+
These scripts are integrated into the GitHub Actions workflow at `.github/workflows/api-diff.yml`:
44+
- **test-branch-logic** job - Runs unit tests
45+
- **api-diff** job - Runs API diff checks with conditional breaking change enforcement
46+
47+
### Branch Naming Convention
48+
The GitHub Actions workflow automatically adjusts its behavior based on branch names:
49+
50+
**Allow Breaking Changes:**
51+
- Any branch containing `breaking` in the name
52+
- Examples: `breaking-api-v2`, `feature-breaking-change`, `api-breaking-update`
53+
- The `--fail-on-breaking` flag is NOT passed to the script
54+
55+
**Fail on Breaking Changes:**
56+
- All other branches (main, master, develop, feature branches, etc.)
57+
- The `--fail-on-breaking` flag IS passed to the script
58+
- Build will fail if breaking changes are detected
59+
60+
This allows developers to explicitly signal when they're working on breaking changes by including `breaking` in their branch name.
61+
62+
## Known Limitations
63+
64+
The oasdiff tool has some non-deterministic behavior due to unordered map iteration in Go:
65+
- **Error counts** (breaking changes) are consistent and reliable
66+
- **Warning counts** may vary by ~2-3% between runs on identical inputs
67+
- This is acceptable for CI purposes as breaking change detection remains accurate
68+
69+
For more details, see the [oasdiff documentation](https://github.com/oasdiff/oasdiff).

‎scripts/api-diff/api-diff.sh‎

Lines changed: 184 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,184 @@
1+
#!/bin/bash
2+
3+
# Script to check API diffs using oasdiff
4+
# Usage: ./scripts/api-diff/api-diff.sh [--fail-on-breaking] [filename.yaml]
5+
# Assumes you have Docker installed and the repo is checked out with master branch available
6+
7+
set -e # Exit on error
8+
set -o pipefail # Catch errors in pipes
9+
10+
# Change to repo root
11+
cd "$(dirname "$0")/../.."
12+
13+
# Configuration
14+
DOCKER_IMAGE="${OASDIFF_DOCKER_IMAGE:-tufin/oasdiff:latest}"
15+
BASE_BRANCH="${BASE_BRANCH:-origin/master}"
16+
17+
FAIL_ON_BREAKING=false
18+
TARGET_FILE=""
19+
DRY_RUN=false
20+
21+
# Parse arguments
22+
for arg in "$@"; do
23+
if [ "$arg" = "--fail-on-breaking" ]; then
24+
FAIL_ON_BREAKING=true
25+
elif [ "$arg" = "--dry-run" ]; then
26+
DRY_RUN=true
27+
elif [[ "$arg" == *.yaml ]]; then
28+
TARGET_FILE="$arg"
29+
fi
30+
done
31+
32+
# If --fail-on-breaking not explicitly set, determine based on branch name
33+
if [ "$FAIL_ON_BREAKING" = false ]; then
34+
CURRENT_BRANCH=${CURRENT_BRANCH:-$(git branch --show-current 2>/dev/null || echo "")}
35+
if [[ "$CURRENT_BRANCH" == *breaking* ]]; then
36+
echo "Branch '$CURRENT_BRANCH' contains 'breaking', allowing breaking changes"
37+
FAIL_ON_BREAKING=false
38+
else
39+
echo "Branch '$CURRENT_BRANCH' does not contain 'breaking', failing on breaking changes"
40+
FAIL_ON_BREAKING=true
41+
fi
42+
fi
43+
44+
if [ "$DRY_RUN" = true ]; then
45+
if [ "$FAIL_ON_BREAKING" = true ]; then
46+
echo "Mode: Failing on breaking changes"
47+
else
48+
echo "Mode: Allowing breaking changes"
49+
fi
50+
echo "Dry run mode, exiting after branch check"
51+
exit 0
52+
fi
53+
54+
echo "Starting API diff check..."
55+
56+
# Ensure we're in the repo root
57+
if [ ! -f "xero_accounting.yaml" ]; then
58+
echo "Error: Not in repo root or xero_accounting.yaml not found"
59+
exit 1
60+
fi
61+
62+
# Fetch master if not already done
63+
git fetch "${BASE_BRANCH%%/*}" "${BASE_BRANCH##*/}" 2>/dev/null || echo "Warning: Could not fetch ${BASE_BRANCH}"
64+
65+
# Create temp directory for master branch files (outside repo to avoid overlap with /current mount)
66+
TEMP_DIR=$(mktemp -d)
67+
trap "rm -rf $TEMP_DIR" EXIT
68+
69+
# Get list of xero*.yaml files (excluding any master_*.yaml files)
70+
if [ -n "$TARGET_FILE" ]; then
71+
# Single file specified
72+
if [ ! -f "$TARGET_FILE" ]; then
73+
echo "Error: File '$TARGET_FILE' not found"
74+
exit 1
75+
fi
76+
files="$TARGET_FILE"
77+
echo "Running diff for single file: $TARGET_FILE"
78+
else
79+
# All xero*.yaml files
80+
files=$(ls xero*.yaml 2>/dev/null | grep -v "^master_")
81+
if [ -z "$files" ]; then
82+
echo "No xero*.yaml files found"
83+
exit 1
84+
fi
85+
fi
86+
87+
BREAKING_CHANGES_FOUND=false
88+
FILES_WITH_BREAKING_CHANGES=()
89+
TOTAL_FILES=0
90+
PROCESSED_FILES=0
91+
92+
echo "========================================"
93+
echo "API Diff Summary"
94+
echo "Using Docker image: $DOCKER_IMAGE"
95+
echo "Base branch: $BASE_BRANCH"
96+
echo "========================================"
97+
98+
for file in $files; do
99+
TOTAL_FILES=$((TOTAL_FILES + 1))
100+
echo ""
101+
echo "========== $file =========="
102+
103+
# Get the file from master branch
104+
if ! git show "$BASE_BRANCH:$file" > "$TEMP_DIR/$file" 2>/dev/null; then
105+
echo "ℹ️ New file (does not exist in master branch)"
106+
continue
107+
fi
108+
109+
# Verify the temp file was created
110+
if [ ! -f "$TEMP_DIR/$file" ]; then
111+
echo "❌ Failed to create temp file"
112+
continue
113+
fi
114+
115+
# Note: oasdiff has some non-deterministic behavior in change counts due to
116+
# unordered map iteration in Go. Error counts are consistent, but warning
117+
# counts may vary by ~2-3% between runs. This is a known limitation.
118+
119+
# Run oasdiff changelog
120+
echo "--- Changelog ---"
121+
set +e
122+
CHANGELOG_OUTPUT=$(docker run --rm -v "$(pwd)":/current -v "$TEMP_DIR":/base "$DOCKER_IMAGE" changelog --include-path-params /base/"$file" /current/"$file" 2>&1)
123+
CHANGELOG_EXIT=$?
124+
set -e
125+
126+
echo "$CHANGELOG_OUTPUT"
127+
128+
if [ $CHANGELOG_EXIT -eq 0 ]; then
129+
echo "✓ Changelog generated successfully"
130+
else
131+
echo "⚠ Could not generate changelog (exit code: $CHANGELOG_EXIT)"
132+
fi
133+
134+
# Run breaking changes check
135+
echo ""
136+
echo "--- Breaking changes check ---"
137+
set +e
138+
BREAKING_OUTPUT=$(docker run --rm -v "$(pwd)":/current -v "$TEMP_DIR":/base "$DOCKER_IMAGE" breaking --fail-on ERR --include-path-params /base/"$file" /current/"$file" 2>&1)
139+
BREAKING_EXIT=$?
140+
set -e
141+
142+
echo "$BREAKING_OUTPUT"
143+
144+
if [ $BREAKING_EXIT -eq 0 ]; then
145+
echo "✓ No breaking changes detected"
146+
else
147+
echo "⚠ Breaking changes detected (exit code: $BREAKING_EXIT)"
148+
BREAKING_CHANGES_FOUND=true
149+
FILES_WITH_BREAKING_CHANGES+=("$file")
150+
fi
151+
152+
PROCESSED_FILES=$((PROCESSED_FILES + 1))
153+
done
154+
155+
echo ""
156+
echo "========================================"
157+
echo "API Diff check completed"
158+
echo "Processed: $PROCESSED_FILES/$TOTAL_FILES files"
159+
echo "========================================"
160+
161+
# Summary
162+
if [ "$BREAKING_CHANGES_FOUND" = true ]; then
163+
echo ""
164+
echo "❌ Breaking changes detected in the following files:"
165+
for file in "${FILES_WITH_BREAKING_CHANGES[@]}"; do
166+
echo " - $file"
167+
# Output GitHub Actions annotation
168+
if [ -n "$GITHUB_ACTIONS" ]; then
169+
echo "::warning file=${file}::Breaking changes detected in this API spec file"
170+
fi
171+
done
172+
173+
if [ "$FAIL_ON_BREAKING" = true ]; then
174+
echo ""
175+
echo "Exiting with error due to breaking changes"
176+
exit 1
177+
else
178+
echo ""
179+
echo "Note: Not failing build (use --fail-on-breaking to fail on breaking changes)"
180+
fi
181+
else
182+
echo ""
183+
echo "✓ No breaking changes detected across all files"
184+
fi

0 commit comments

Comments
 (0)