Establish parallel-agent workflow guidelines for shared checkout/branch integration

[cmungall]

Summary

We need explicit, repository-level guidelines for multiple agents working in parallel against the same checkout/branch family. Recent PR work surfaced avoidable complexity and CI regressions caused by branch divergence, dirty working trees, and conflict-resolution flow under non-force-push constraints.

This issue proposes a lightweight but enforceable workflow for agent collaboration so that curation changes remain safe, reviewable, and mergeable.

Case Study Context

A recent supports-alignment effort across many disorder YAMLs required conflict resolution after upstream merges landed in parallel.

Observed sequence (condensed):

• A large curation PR was created from a branch with substantial local dirt and branch drift.
• Upstream moved quickly (multiple merged PRs), introducing many file conflicts in overlapping disorder YAMLs.
• Conflict resolution was performed in temporary worktrees, but push strategy constraints (no force push in execution environment) complicated branch updates.
• A fix commit for kb/disorders/Chronic_Myeloid_Leukemia.yaml (removing schema-incompatible target_mechanisms) was temporarily present, then effectively regressed at PR head after merge-history manipulation.
• CI correctly failed on schema validation:
  • Additional properties are not allowed ('target_mechanisms' was unexpected) in /treatments/*
• Final resolution required a clean worktree from exact PR head, replaying the fix, validating, and fast-forward pushing.

Net effect: the technical content change was small, but process overhead was high.

Problem Statement

Parallel agents currently lack a shared, explicit operating contract for:

• Branch ownership and synchronization strategy
• Dirty workspace handling
• Conflict resolution and push rules
• Post-push verification against exact remote head

Without this, we get increased risk of:

• Reintroducing reverted/fixed content during conflict resolution
• CI churn and delayed merges
• Hard-to-audit branch histories
• Confusion between local branch state and actual PR head state

Desired Action

Create and adopt a "Parallel Agent Workflow" guideline for this repo.

Proposed Deliverables

1. Add docs/parallel-agent-workflow.md (or equivalent) covering:
   • Required use of clean worktrees for PR surgery/conflict resolution
   • Branch naming/ownership conventions for concurrent agents
   • Rules for handling dirty checkouts (never operate directly on shared dirty tree for integration)
   • Preferred rebase/merge strategy under force-push restrictions
   • Scope isolation for curation PRs (e.g., avoid cache/page churn unless intentional)
2. Add a short checklist to PR template or contributor docs:
   • "Verified PR head SHA"
   • "Validated risky files from exact remote head"
   • "Ran just validate/just validate-all from clean worktree"
3. Add a guardrail check where feasible:
   • Schema-incompatible key linting for known hotspots (e.g., unsupported treatment keys)
4. Add a "conflict recovery" playbook:
   • Step-by-step for reconstructing a mergeable branch without reintroducing stale content

Acceptance Criteria

• A documented, discoverable workflow exists and is linked from CONTRIBUTING.md and/or CLAUDE.md.
• PR checklist includes remote-head verification and clean-worktree validation.
• At least one lightweight automated guardrail is in place or explicitly deferred with rationale.
• One trial run with agent-based curation follows the new workflow and reports outcome.

Why This Matters

This repo increasingly uses agent-assisted curation and concurrent PR activity. A clear protocol will reduce integration risk, shorten merge time, and make failures easier to diagnose when they do occur.

Signed,
codex

[github-actions]

Comprehensive Analysis: Issue #360

📋 Summary

This issue proposes establishing formal parallel-agent workflow guidelines for the dismech repository to address coordination challenges when multiple AI agents work concurrently on overlapping disorder YAML files. The motivation comes from a recent conflict-heavy PR (#357, supports-alignment effort) that exposed gaps in branch management, conflict resolution, and validation protocols.

---

🎯 Context & Motivation

Triggering Event: PR #357 (curate/supports-alignment-disorders-2026-02-16-clean, merged via ca4df7d) involved large-scale curation across many disorder YAMLs. The PR experienced:

• Branch divergence: Local branch drifted from upstream as multiple PRs merged in parallel
• Dirty working tree: Substantial uncommitted changes complicated integration
• Merge conflict complexity: Overlapping disorder YAML edits required manual resolution
• Regression incident: A fix to kb/disorders/Chronic_Myeloid_Leukemia.yaml (removing schema-incompatible target_mechanisms field) was temporarily present, then regressed after merge-history manipulation
• CI failure: Schema validation correctly caught the regression: Additional properties are not allowed ('target_mechanisms' was unexpected) in /treatments/*
• High process overhead: Small content change required extensive worktree surgery, SHA verification, and multi-step recovery

Core Problem: Multiple agents lack a shared operating contract for concurrent branch work, leading to avoidable rework, CI churn, and merge delays.

---

🔗 Related Issues

This issue intersects with several ongoing challenges:

1. cache conflicts #337 - Cache conflicts (open)

   • Links to linkml-term-validator#9
   • Frequent cache-related merge conflicts from parallel agents
   • Connection: Scope isolation (proposed in Establish parallel-agent workflow guidelines for shared checkout/branch integration #360 deliverable Bump actions/checkout from 4 to 6 #1) would help avoid unintentional cache churn

2. Epic: plan for agent provenance on yamls #68 - Epic: Agent provenance on YAMLs (open)

   • Tracks which agents/tools modified which files
   • Proposes ai4curation/ai-blame integration
   • Connection: Provenance tracking would make conflict attribution clearer during parallel work

3. Add more instructions for new agent managers #190 - Add more instructions for new agent managers (open, assigned to @sujaypatil96)

   • Improve onboarding for human curators managing AI agents
   • Calls for better documentation in README.md, CONTRIBUTING.md, and .claude/commands/curate.md
   • Connection: Parallel-agent workflow docs (proposed in Establish parallel-agent workflow guidelines for shared checkout/branch integration #360) would complement agent manager onboarding

---

📦 Proposed Deliverables

The issue specifies four concrete deliverables:

1. New Documentation: docs/parallel-agent-workflow.md

Should cover:

• ✅ Clean worktree requirement: Never perform PR surgery/conflict resolution on dirty checkouts
• ✅ Branch naming/ownership conventions: Clear rules for concurrent agent branches (e.g., curate/<disorder>-<date>-<agent-id>)
• ✅ Dirty checkout handling: Isolate work via git worktree or stash/commit before integration
• ✅ Rebase/merge strategy: Preferred approaches when force-push is restricted
• ✅ Scope isolation: Avoid unintentional cache/page regeneration unless explicitly part of the PR scope

Placement: docs/ directory exists and has precedent (docs/embeddings.md, docs/explanation/), so docs/parallel-agent-workflow.md fits naturally.

2. PR Template/Contributor Checklist Updates

Add verification steps:

• Verified PR head SHA matches remote
• Validated risky files from exact remote head (not local dirty state)
• Ran just validate / just validate-all from clean worktree

Implementation: Update CONTRIBUTING.md (already exists) and/or add .github/pull_request_template.md.

3. Automated Guardrail for Known Hotspots

Suggests schema-incompatible key linting for common errors (e.g., target_mechanisms in treatment blocks, which caused the #357 regression).

Options:

• Custom pre-commit hook
• Enhanced LinkML validation rule
• GitHub Actions check

4. Conflict Recovery Playbook

Step-by-step guide for reconstructing mergeable branches after conflicts without reintroducing stale content.

Example structure:

## Conflict Recovery Playbook

### Scenario: Upstream Merged While Your PR is Open

1. **Fetch latest upstream:**
   ```bash
   git fetch origin main

2. Create clean worktree:

   git worktree add ../dismech-conflict-resolution origin/your-branch-name
   cd ../dismech-conflict-resolution

3. Rebase/merge and resolve conflicts:

   git rebase origin/main
   # or: git merge origin/main

4. Validate from clean state:

   just validate-all

5. Verify exact remote head:

   git rev-parse HEAD  # Note this SHA
   git push origin your-branch-name

6. On GitHub: Check CI passes against pushed SHA

---

### ✅ Acceptance Criteria

From the issue:
- [ ] Documented workflow exists and is linked from `CONTRIBUTING.md` and/or `CLAUDE.md`
- [ ] PR checklist includes remote-head verification and clean-worktree validation
- [ ] At least one automated guardrail is in place (or explicitly deferred with rationale)
- [ ] Trial run: One agent-based curation follows the new workflow and reports outcomes

---

### 🛠️ Implementation Suggestions

#### Documentation Strategy

**Primary:** Create `docs/parallel-agent-workflow.md` as the canonical reference.

**Integration points:**
- **CONTRIBUTING.md**: Add section "Working with Multiple Agents" linking to the new doc
- **CLAUDE.md**: Add reminder under "Pull Request Process" section pointing agents to parallel workflow guidelines
- **.claude/skills/**: Consider a new `parallel-workflow/` skill with checklist steps

#### Automated Guardrails: Prioritization

Given the #357 regression involved `target_mechanisms` appearing in treatment blocks:

**Quick win:**
```bash
# Pre-commit hook or CI check
uv run python -m dismech.validate --strict-treatment-keys kb/disorders/*.yaml

Could detect unexpected keys in treatment descriptors before merge.

Broader approach: Enhance LinkML validation with closed: true constraints on treatment slot ranges to make the schema itself reject extra properties more explicitly.

Trial Run Candidate

Next large-scale curation effort (e.g., adding MAXO treatment terms across multiple disorders, or aligning with external resources per #333) could serve as the trial run following the new workflow.

---

📚 Additional Context

Current Infrastructure:

• ✅ Validation stack: just qc runs linkml-validate, linkml-term-validator, linkml-reference-validator
• ✅ Clean worktree precedent: The Curation: align supports levels across disorder evidence #357 resolution did ultimately use clean worktrees (mentioned in issue description)
• ✅ GitHub AI integrations: ai4curation/github-ai-integrations enables agent-driven PRs
• ⚠️ Gap: No formal documentation of these patterns yet

Relevant External Resources:

• Git worktree documentation: https://git-scm.com/docs/git-worktree
• Parallel development patterns: Common in multi-agent systems and CI/CD contexts
• ai4curation ecosystem: https://ai4curation.io/ (provenance, agent eval frameworks)

---

🎬 Recommended Next Steps

1. Assign ownership: Designate someone (human or agent) to draft docs/parallel-agent-workflow.md

2. Gather examples: Document the Curation: align supports levels across disorder evidence #357 resolution sequence as a concrete case study

3. Stakeholder input: Get feedback from frequent contributors (@cmungall, @sujaypatil96, active agent managers)

4. Incremental rollout:

   • Start with documentation (deliverable Bump actions/checkout from 4 to 6 #1)
   • Add PR checklist (deliverable Bump actions/setup-node from 3 to 6 #2)
   • Defer complex guardrails (deliverable Bump astral-sh/setup-uv from 5 to 7 #3) unless low-effort option emerges
   • Build conflict playbook (deliverable Bump actions/github-script from 6 to 8 #4) from real recovery experiences

5. Trial period: Apply new workflow to next 3-5 PRs and iterate based on friction points

---

🏷️ Current Labels

• documentation ✅ (primary deliverable is docs)
• enhancement ✅ (workflow improvement)
• infra ✅ (affects repository processes/tooling)

Suggested additions:

• None needed - current labels accurately reflect the issue scope

---

💡 Why This Matters (from issue)

This repo increasingly uses agent-assisted curation and concurrent PR activity. A clear protocol will reduce integration risk, shorten merge time, and make failures easier to diagnose when they do occur.

Impact metrics to track:

• Time-to-merge for multi-disorder PRs
• Frequency of schema validation failures in CI
• Number of conflict resolution cycles per PR
• Agent manager satisfaction with coordination clarity

---

Signed,
Claude (Sonnet 4.5) - Issue Analysis Agent