Skip to content

Commit 0da6651

Browse files
jeqoclaude
jeqo
and
claude
committed
docs(inkless:sync): add entry point and update README
- SYNC-PROMPT.md: Simple entry point for AI-assisted sync sessions - README.md: Comprehensive documentation for all sync scripts - docs/inkless/README.md: Add link to sync tooling Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
1 parent 633b659 commit 0da6651

File tree

3 files changed

+331
-0
lines changed

3 files changed

+331
-0
lines changed

‎docs/inkless/README.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,9 @@ See [Releases](RELEASES.md) for available versions.
4949
- [Releases](RELEASES.md) - Artifacts released (binaries and docker images)
5050
- [Glossary](GLOSSARY.md) - Definitions of Inkless-specific terms and concepts
5151

52+
### Development & Maintenance
53+
- [Upstream Sync Tooling](../../inkless-sync/README.md) - Scripts for syncing with Apache Kafka upstream
54+
5255
> **Note:** Configuration and metrics documentation is provided in reStructuredText (.rst) format as it is auto-generated by Kafka's documentation tooling. Other documentation uses Markdown (.md) for easier editing.
5356
5457
### Important Notes About Auto-Generated Documentation

‎inkless-sync/README.md‎

Lines changed: 293 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,293 @@
1+
# Inkless Upstream Sync Tooling
2+
3+
This directory contains scripts and configuration for syncing inkless with upstream Apache Kafka.
4+
5+
## Overview
6+
7+
The sync process follows the [Versioning Strategy](../docs/inkless/VERSIONING-STRATEGY.md):
8+
- Uses **merge commits** (not rebase) for velocity
9+
- Preserves inkless-specific features and configurations
10+
- Creates structured commits for different types of adaptations
11+
12+
## AI-Assisted Sync (Recommended)
13+
14+
Start a sync session with Claude Code using **[SYNC-PROMPT.md](SYNC-PROMPT.md)**.
15+
16+
The agent will guide you to the appropriate workflow:
17+
- **Main Sync** → [MAIN-SYNC-PROMPT.md](MAIN-SYNC-PROMPT.md)
18+
- **Release Sync** → [RELEASE-SYNC-PROMPT.md](RELEASE-SYNC-PROMPT.md)
19+
20+
## Scripts Overview
21+
22+
| Script | Purpose |
23+
|--------|---------|
24+
| `main-sync.sh` | Sync main branch with Apache Kafka trunk |
25+
| `release-sync.sh` | Sync release branches with upstream patch releases |
26+
| `sync-status.sh` | Check how far behind branches are from upstream |
27+
| `branch-consistency.sh` | Check if inkless commits from main are in release branches |
28+
| `create-release-branch.sh` | Create new inkless release branches |
29+
| `cherry-pick-to-release.sh` | Cherry-pick inkless commits to release branches |
30+
31+
## Two Types of Sync
32+
33+
| Type | Branch | Script | Use Case |
34+
|------|--------|--------|----------|
35+
| **Main Sync** | `main` | `main-sync.sh` | Weekly/biweekly sync with Apache Kafka trunk |
36+
| **Release Sync** | `inkless-4.0`, etc. | `release-sync.sh` | Sync release branches with upstream patch releases |
37+
38+
### When to Use Each
39+
40+
- **Main Sync**: Regular development sync to keep up with Apache Kafka trunk
41+
- **Release Sync**: When Apache releases a patch (e.g., 4.0.1) and we need to incorporate fixes into our release branch
42+
43+
## Quick Start
44+
45+
### Regular Weekly/Biweekly Sync
46+
47+
```bash
48+
# Sync with latest apache/kafka trunk
49+
./inkless-sync/main-sync.sh
50+
```
51+
52+
### Sync Before New Kafka Version
53+
54+
```bash
55+
# Sync to the commit just before a version tag (e.g., before 4.3)
56+
./inkless-sync/main-sync.sh --before-version 4.3
57+
```
58+
59+
### Dry Run (Preview)
60+
61+
```bash
62+
# See what would happen without making changes
63+
./inkless-sync/main-sync.sh --dry-run
64+
```
65+
66+
### Release Branch Sync
67+
68+
```bash
69+
# List available upstream release tags
70+
./inkless-sync/release-sync.sh inkless-4.0 --list-tags
71+
72+
# Sync inkless-4.0 to Apache Kafka 4.0.1
73+
./inkless-sync/release-sync.sh inkless-4.0 --to-tag 4.0.1
74+
```
75+
76+
For detailed release sync workflow, see [RELEASE-SYNC-GUIDE.md](RELEASE-SYNC-GUIDE.md).
77+
78+
### Check Sync Status
79+
80+
```bash
81+
# Check how far behind main is from upstream trunk
82+
./inkless-sync/sync-status.sh main
83+
84+
# Check release branch status
85+
./inkless-sync/sync-status.sh inkless-4.0
86+
87+
# Check all branches
88+
./inkless-sync/sync-status.sh --all
89+
```
90+
91+
### Branch Consistency Check
92+
93+
Check if inkless commits from main have been cherry-picked to release branches:
94+
95+
```bash
96+
# Check inkless-4.0 consistency with main
97+
./inkless-sync/branch-consistency.sh inkless-4.0
98+
99+
# Show missing commits with cherry-pick commands
100+
./inkless-sync/branch-consistency.sh inkless-4.0 --missing
101+
102+
# Show ALL missing commits (including old ones that were intentionally skipped)
103+
./inkless-sync/branch-consistency.sh inkless-4.0 --missing --all
104+
```
105+
106+
### Create New Release Branch
107+
108+
```bash
109+
# Create inkless-4.2 from main (main must be at 4.2.0-inkless-SNAPSHOT or later)
110+
./inkless-sync/create-release-branch.sh 4.2
111+
112+
# Preview what would happen
113+
./inkless-sync/create-release-branch.sh 4.2 --dry-run
114+
115+
# Force creation even if version doesn't match
116+
./inkless-sync/create-release-branch.sh 4.2 --force
117+
```
118+
119+
After creating the branch, sync with upstream to set the release version:
120+
```bash
121+
./inkless-sync/release-sync.sh inkless-4.2 --to-tag 4.2.0
122+
```
123+
124+
### Cherry-pick to Release Branches
125+
126+
```bash
127+
# Cherry-pick all missing inkless commits to inkless-4.0
128+
./inkless-sync/cherry-pick-to-release.sh inkless-4.0
129+
130+
# Preview what would be cherry-picked
131+
./inkless-sync/cherry-pick-to-release.sh inkless-4.0 --dry-run
132+
133+
# Cherry-pick specific commits
134+
./inkless-sync/cherry-pick-to-release.sh inkless-4.0 abc123 def456
135+
```
136+
137+
## How It Works (Main Sync)
138+
139+
### Phase 1: Preparation
140+
1. Fetches upstream apache/kafka
141+
2. Determines sync target (trunk HEAD or before-version)
142+
3. Generates "inkless manifest" - list of files we've modified
143+
4. Creates sync branch: `sync/upstream-YYYYMMDD`
144+
145+
### Phase 2: Merge
146+
1. Executes `git merge` from upstream
147+
2. Categorizes conflicts:
148+
- **Protected**: Files in `storage/inkless/`, `docs/inkless/`, etc. → Use "ours"
149+
- **Auto-resolvable**: Import conflicts, trivial changes → Auto-resolve
150+
- **Manual**: Complex conflicts → Stop and ask for human help
151+
3. Completes merge commit or reports conflicts needing attention
152+
153+
### Phase 3: Adaptation
154+
1. Compiles the codebase
155+
2. Analyzes any compilation errors
156+
3. Errors should be fixed and committed as: `sync(compile): fix compilation errors`
157+
158+
### Phase 4: Verification
159+
1. Runs inkless-specific tests
160+
2. Verifies inkless features are preserved (manifest check)
161+
3. Generates sync report
162+
163+
## File Structure
164+
165+
```
166+
inkless-sync/
167+
├── main-sync.sh # Main branch sync script
168+
├── release-sync.sh # Release branch sync script
169+
├── sync-status.sh # Check sync status vs upstream
170+
├── branch-consistency.sh # Check cherry-pick consistency
171+
├── create-release-branch.sh # Create new release branches
172+
├── cherry-pick-to-release.sh # Cherry-pick commits to releases
173+
├── README.md # This file
174+
├── SYNC-PROMPT.md # Entry point prompt for AI-assisted sync
175+
├── RELEASE-SYNC-GUIDE.md # Release sync documentation
176+
├── RELEASE-SYNC-PROMPT.md # AI prompt for release syncs
177+
├── CONFLICT-RESOLUTION-STRATEGY.md # Conflict resolution guidance
178+
├── MAIN-SYNC-PROMPT.md # AI prompt for main branch syncs
179+
├── MAIN-SYNC-SESSION-TEMPLATE.md # Session template for main syncs
180+
├── MAIN-SYNC-ACTION-PLAN.md # Action plan for main syncs
181+
├── RELEASE-SYNC-SESSION-TEMPLATE.md # Session template for release syncs
182+
├── RELEASE-SYNC-ACTION-PLAN.md # Action plan for release syncs
183+
├── lib/
184+
│ └── common.sh # Shared utility functions
185+
└── config/
186+
└── protected-patterns.txt # Files to protect during merge
187+
```
188+
189+
During sync, a `.sync/` directory is created with:
190+
```
191+
.sync/
192+
├── sync-info.txt # Sync metadata
193+
├── manifest-files.txt # Files modified by inkless
194+
├── manifest-stats.txt # Diff stats from merge base
195+
├── conflicted-files.txt # Files with conflicts
196+
├── conflicts-protected.txt # Protected file conflicts
197+
├── conflicts-auto-resolvable.txt # Auto-resolvable conflicts
198+
├── conflicts-manual.txt # Conflicts needing manual resolution
199+
├── compile-output.txt # Compilation output
200+
├── compile-errors.txt # Extracted compilation errors
201+
└── SYNC-REPORT.md # Final sync report
202+
```
203+
204+
## Configuration
205+
206+
### Protected Patterns (`config/protected-patterns.txt`)
207+
208+
Files matching these patterns will prefer "ours" (inkless version) during conflicts:
209+
210+
```
211+
# Always protect
212+
storage/inkless/**
213+
docs/inkless/**
214+
.github/workflows/inkless*.yml
215+
gradle.properties
216+
217+
# Review carefully
218+
server-common/src/main/java/org/apache/kafka/server/config/ServerConfigs.java
219+
```
220+
221+
### Adding Apache Remote
222+
223+
If you don't have the apache remote configured:
224+
225+
```bash
226+
git remote add apache https://github.com/apache/kafka.git
227+
git fetch apache
228+
```
229+
230+
## Commit Convention
231+
232+
After sync, use these commit message prefixes:
233+
234+
| Prefix | Description |
235+
|--------|-------------|
236+
| `merge:` | The merge commit itself |
237+
| `sync(compile):` | Fixing compilation errors from API changes |
238+
| `sync(test):` | Fixing test infrastructure changes |
239+
| `sync(config):` | Preserving inkless configurations |
240+
| `sync(verify):` | Verification and manifest updates |
241+
242+
Example:
243+
```bash
244+
git commit -m "sync(compile): adapt to KafkaMetricsGroup constructor change"
245+
git commit -m "sync(test): add NoOpRemoteLogMetadataManager to test config"
246+
```
247+
248+
## Troubleshooting
249+
250+
### Manual Conflicts
251+
252+
If the script reports manual conflicts:
253+
254+
1. Check `.sync/conflicts-manual.txt` for the list
255+
2. Resolve each conflict manually
256+
3. `git add <resolved-files>`
257+
4. `git commit` to complete the merge
258+
5. Continue with adaptation phase
259+
260+
### Compilation Errors
261+
262+
1. Check `.sync/compile-errors.txt` for error list
263+
2. Fix errors in logical groups
264+
3. Commit each group: `git commit -m "sync(compile): <description>"`
265+
266+
### Test Failures
267+
268+
1. Check test output in `.sync/test-*-output.txt`
269+
2. Fix failures
270+
3. Commit: `git commit -m "sync(test): <description>"`
271+
272+
## Agent Usage
273+
274+
This script is designed to be run by an automated agent (e.g., Claude):
275+
276+
1. Agent runs `./inkless-sync/main-sync.sh`
277+
2. If manual conflicts: Agent reports to human via PR/issue
278+
3. If compilation errors: Agent attempts fixes, commits separately
279+
4. If test failures: Agent analyzes and fixes or reports
280+
5. Agent creates PR with sync report
281+
282+
The structured commit approach makes it easy to:
283+
- Review what changed and why
284+
- Revert specific fixes if needed
285+
- Understand the sync process
286+
287+
## Related Documentation
288+
289+
- [Versioning Strategy](../docs/inkless/VERSIONING-STRATEGY.md)
290+
- [Inkless README](../docs/inkless/README.md)
291+
- [Release Sync Guide](RELEASE-SYNC-GUIDE.md)
292+
- [Conflict Resolution Strategy](CONFLICT-RESOLUTION-STRATEGY.md)
293+
- [Main Sync Prompt](MAIN-SYNC-PROMPT.md)

‎inkless-sync/SYNC-PROMPT.md‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
# Inkless Sync Prompt
2+
3+
Copy and paste this prompt to start a sync session:
4+
5+
---
6+
7+
```
8+
I need to sync inkless with upstream Apache Kafka.
9+
10+
## Available Sync Types
11+
12+
1. **Main Sync** - Sync the `main` branch with Apache Kafka trunk
13+
- Use for: Weekly/biweekly development sync
14+
- Prompt: `inkless-sync/MAIN-SYNC-PROMPT.md`
15+
16+
2. **Release Sync** - Sync a release branch (e.g., `inkless-4.0`) with upstream patches
17+
- Use for: When Apache releases a patch (e.g., 4.0.1)
18+
- Prompt: `inkless-sync/RELEASE-SYNC-PROMPT.md`
19+
20+
## Context
21+
22+
- Scripts are in `inkless-sync/`
23+
- Session templates track progress in `.sync/`
24+
- Action plans have file-by-file playbooks
25+
26+
Please help me determine which sync type I need and guide me through the process.
27+
Read the appropriate prompt file for detailed instructions.
28+
```
29+
30+
---
31+
32+
The agent will:
33+
1. Ask which sync type you need (or infer from context)
34+
2. Read the appropriate detailed prompt
35+
3. Guide you through the sync process

0 commit comments

Comments
 (0)