-
Notifications
You must be signed in to change notification settings - Fork 1
enhance case and experiment templates and README #584
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,170 +1,171 @@ | ||
| --- | ||
| name: Case | ||
| about: Product-level description in stakeholder language - defines system boundaries and value propositions | ||
| name: Case Brief | ||
| about: Product-level overview for stakeholders - defines engineered system and acceptance criteria | ||
| title: '[CASE] [Brief descriptive title]' | ||
| labels: 'case' | ||
| assignees: '' | ||
|
|
||
| --- | ||
|
|
||
| [Case writing rules: | ||
| [Case Brief writing rules: | ||
| - Product Focus: Define agent value and experience, not technical implementation | ||
| - Problem Analysis: Lives in Case Product Specification - Brief references Spec for context | ||
| - Agent Priority: List agents by % importance with Human/System distinction (even if similar to other cases) | ||
| - Target System Clarity: Explicitly identify target system and distinguish from consumer/dependency systems | ||
| - System Boundaries: Explicitly state what's included/excluded from this case | ||
| - Basic English: Write for non-native speakers, avoid complex technical terms | ||
| - Scope Limit: Target achievable milestones of 3–6 months only | ||
| - Agent-Driven: Focus on agent behaviour and adoption, rather than system performance] | ||
|
|
||
| ## Target System | ||
|
|
||
| [Which system does this case address and what is its position in the larger system architecture?] | ||
|
|
||
| ## Problem Statement | ||
|
|
||
| [Describe the agent/business problem this case solves. What needs to change in the current state and why? Focus on WHAT agents need and WHY it matters. Leave technical details for `hypotheses` of experiments] | ||
|
|
||
| ### Current Agent Experience | ||
| - Stakeholder Language: This brief is for business/product stakeholders | ||
| - Minimal Content: Engineers need context to understand "what" and "why", not extensive product analysis | ||
| - System Boundaries: Explicitly state what's included/excluded | ||
| - Link to Details: Extended analysis lives in Coda, link from here | ||
| - Scenario-Driven: Focus on agent behavior and acceptance, not system performance | ||
| - Scope Limit: Target 3-6 month achievable milestones | ||
| - Experiment Mapping: Link acceptance criteria to implementing experiments | ||
| - Metrics Cascade: Case Product Specification defines product success metrics → Case Brief translates to verifiable acceptance criteria → Experiments validate technical implementation | ||
| - Link Don't Duplicate: Reference specifications, don't copy content] | ||
|
|
||
| [What agents experience today that needs improvement] | ||
| ## Engineered System | ||
|
|
||
| ### Desired Agent Experience | ||
| [Which specific system/subsystem are we building and what is its position in the larger system architecture?] | ||
|
|
||
| [What agents should be able to do after this case is complete] | ||
| *For detailed system context, complete problem analysis including current/desired agent experience, see: [Case Product Specification](link-to-coda)* | ||
|
|
||
| ## Value Proposition | ||
| ## Agent Priority Overview | ||
|
|
||
| [Clear business / agent value that this case provides] | ||
| [High-level stakeholder priorities with percentages. Detailed analysis in Case Product Specification in Coda.] | ||
|
|
||
| ## Agent Analysis | ||
| **Priority Distribution:** [e.g., "Primary: 60% Developers; Secondary: 30% System Integrators; Minimal: 10% End Users"] | ||
|
|
||
| [Map all agents (human and system) by priority with percentages. Focus on WHO / WHAT will interact with or benefit from the Target System] | ||
| **Agent Priority Overview**: [e.g., "Primary: 60% Developers; Secondary: 30% Monitoring Systems; Minimal: 10% End Users"] | ||
| [Optional: Include an evaluation / justification for why these priorities make sense for this case] | ||
| **Rationale:** *Optional* [1-2 sentence justification for these priorities] | ||
|
|
||
| ### [Primary Agent Name] ([X%] - Primary) | ||
| - **Agent Type**: [Human Agent: role / persona] OR [System Agent: system type / function] | ||
| - **Current Pain Points**: [What problems do they have today with existing systems] | ||
| - **Desired Outcomes**: [What success looks like for them] | ||
| - **Agent Journey**: [Action] → [Action] → [Successful Outcome] | ||
| - **Success Metrics**: [How they measure success - can include system metrics for System Agents] | ||
| - **Integration Requirements**: [For System Agents: APIs, data formats, protocols needed] | ||
|
|
||
| ### [Secondary Agent Name] ([Y%] - Secondary) | ||
|
|
||
| [Same structure as above] | ||
|
|
||
| [Continue the pattern for all Agents, ordered by priority] | ||
| *For detailed agent analysis with agent journeys and integration requirements, see: [Case Product Specification](link-to-coda)* | ||
|
|
||
| ## Expected Agent Experience & Acceptance | ||
|
|
||
| [BDD scenarios that define both the Target System behaviour and the acceptance criteria. Describe what agents will experience, NOT how the Target System works internally. Focus on acceptance testing, not repeating the desired outcomes already listed in Agent Analysis. Validation priorities are derived from Agent Priority Overview above – no separate priority statement needed here] | ||
| [Define scenarios the Engineered System must handle and acceptance criteria. Focus on observable outcomes, not internal system operations.] | ||
|
|
||
| *Note: Link acceptance criteria to implementing experiments during experiment planning phase.* | ||
|
|
||
| ### Agent Acceptance Scenarios | ||
| **Scenario 1: [Primary Happy Path for Human Agent]** | ||
| - Given [agent context / starting point] | ||
|
|
||
| **Scenario 1: [Primary Scenario for Primary Agent]** | ||
| - Given [detailed initial conditions] | ||
| - When [agent performs action] | ||
| - Then [agent experiences result] | ||
| - And [additional agent benefit] | ||
|
|
||
| **Acceptance Criteria:** | ||
| [Each criterion should be demonstrable within 5-10 minutes by non-developers or through developer demo. Validation methods: Observable (UI/logs/behavior), Measurable (counted/timed), Testable (test scripts), User-Validated (actual users)] | ||
|
|
||
| [It would be preferable if non-developers can verify this work in 5-10 minutes] | ||
| - [ ] [Specific measurable criterion for this scenario] | ||
| - [ ] [Performance / quality requirement for this behavior] | ||
| - [ ] [Specific criterion] → **Experiment**: [Link #XXX when available or TBD] | ||
| - *Validation: [How to verify - e.g., "Dashboard shows metric within target"]* | ||
| - [ ] [Performance/quality requirement] → **Experiment**: [Link #XXX when available or TBD] | ||
| - *Validation: [Verification method]* | ||
|
|
||
| **Scenario 2: [Primary Happy Path for System Agent]** | ||
|
|
||
| - Given [system agent needs specific data / functionality] | ||
| - When [system agent makes API call / integration request] | ||
| - Then [target system provides required response / data] | ||
| - And [system agent can successfully complete its function] | ||
| **Scenario 2: [Secondary Scenario - Success path for Secondary Agent]** | ||
| - Given [different initial conditions] | ||
| - When [alternative agent action] | ||
| - Then [expected alternative outcome] | ||
|
|
||
| **Acceptance Criteria:** | ||
| - [ ] [Specific criterion] → **Experiment**: [Link #XXX when available or TBD] | ||
| - *Validation: [Verification method]* | ||
|
|
||
| [How to verify system agent integration works, e.g. API tests, data format checks] | ||
| **Scenario 3: [Alternative Scenario - Different approach or edge case]** | ||
| - Given [edge case conditions] | ||
| - When [action that triggers alternative path] | ||
| - Then [expected handling] | ||
|
|
||
| **Scenario 3: [Alternative Path]** | ||
| **Acceptance Criteria:** | ||
| - [ ] [Specific criterion] → **Experiment**: [Link #XXX when available or TBD] | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. not clear why we need a backlink to experiment here |
||
| - *Validation: [Verification method]* | ||
|
|
||
| Given [Different initial conditions] | ||
| When [Alternative stakeholder action] | ||
| Then [Expected alternative response] | ||
| **Scenario 4: [Error Scenario - Failure case and recovery]** | ||
| - Given [error conditions] | ||
| - When [action that triggers error] | ||
| - Then [expected error handling and recovery] | ||
|
|
||
| **Acceptance Criteria:** | ||
| - [ ] [Error handling criterion] → **Experiment**: [Link #XXX when available or TBD] | ||
| - *Validation: [Verification method]* | ||
|
|
||
| - [ ] [Specific measurable criterion for this scenario] | ||
| ## Scope Summary | ||
|
|
||
| **Scenario 4: [Error/Edge Case Handling]** | ||
| ### Engineered System Scope | ||
|
|
||
| Given [Error conditions] | ||
| When [Action that triggers error] | ||
| Then [Expected error handling behavior] | ||
| **In Scope:** [What this system will do - boundaries included] | ||
|
|
||
| **Acceptance Criteria:** | ||
| **Out of Scope:** [What explicitly will not be addressed - link to other cases handling these] | ||
|
|
||
| - [ ] [Specific measurable criterion for error handling] | ||
| *For detailed interfaces and integration points, see: [Case Architecture Specification](link-to-arch-doc)* | ||
|
|
||
| ## System Context & Boundaries | ||
| ## Critical Dependencies & Blockers | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. objection: there are tags and comments for blockers |
||
|
|
||
| ### Target System Scope | ||
| **Blocking This Case:** | ||
| - [Case/System #X]: [What must complete before we can proceed] | ||
|
|
||
| In Scope: [What the Target System will do and the boundaries included] | ||
| Out of Scope: [What explicitly will not be addressed] | ||
| Interfaces: [External systems (consumer systems, dependency systems, peer / interacting systems), data flows, and dependencies] | ||
| **This Case Blocks:** | ||
| - [Case/System #Y]: [What depends on this case's completion] | ||
|
|
||
| ### Quality Attributes | ||
| **Bottlenecks** (Resource constraints): | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. objection: this is an architectural question. The whole section might be moved to architecture. |
||
| - [Resource constraint] - Impact: [Description] | ||
|
|
||
| Performance: [Response time, throughput requirements] | ||
| Scalability: [Growth expectations and constraints] | ||
| Reliability: [Uptime, error rate expectations] | ||
| Security: [Security requirements and compliance needs] | ||
| Usability: [User experience requirements] | ||
| **External Blockers** (Third-party dependencies): | ||
| - [Third-party dependency] - Expected resolution: [Timeline] | ||
|
|
||
| ### Constraints & Dependencies: | ||
| **Critical Path Items:** | ||
| - [Dependency with resolution date] | ||
| - [Risk requiring immediate attention] | ||
|
|
||
| Dependencies: [External dependencies and other cases / systems this depends on] | ||
| Technical Constraints: [Technical limitations and requirements] | ||
| Business Constraints: [Business rules, resource, and timeline constraints] | ||
| *For complete dependency analysis and technical interfaces, see: [Case Product Specification](link-to-coda) and [Case Architecture Specification](link-to-arch-doc)* | ||
|
|
||
| ## Risks Assessment | ||
| ## Decision Log | ||
|
|
||
| **[Risk 1]** | ||
| [Enumeration of related ADRs - decisions themselves live in ADR documents] | ||
|
|
||
| Impact: [High / Med / Low] | ||
| Probability: [High / Med / Low] | ||
| Mitigation Strategy: [Mitigation approach] | ||
| Owner: [Responsible person] | ||
| - [Date] - ADR #[XXXX] - [Case decomposition decision] - [Link to ADR] | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. you don't need backlinks if in each ADR you will mention this case. Then github issue will mention the links below. It's an overengineering: it will be a pain in the ass to fill this doc with so many requirements. |
||
| Status: [Active/Superseded by ADR #[XXXX]] | ||
| - [Date] - ADR #[XXXX] - [Brief description] - [Link to ADR] | ||
| Status: [Active/Superseded by ADR #[XXXX]] | ||
|
|
||
| ## Decision Log | ||
| ## References & Links | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. not needed, everyone can add the relevant info optionally |
||
|
|
||
| [Record key architectural and design decisions] | ||
| **Full Case Details:** | ||
| - [Case Product Specification](link-to-coda) - Extended product analysis, detailed agent journeys, business context | ||
|
|
||
| [Date] - [Decision] - [Rationale] - [Impact on agents] | ||
| Status: [Active/Superseded] | ||
| **Related Architecture:** | ||
| - [Case Architecture Specification](link-to-arch-doc) - Technical architecture, interfaces, integration points | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. repeated |
||
|
|
||
| ## Learning Outcomes | ||
|
|
||
| [To be filled in during and after the case has been completed] | ||
| [To be filled during and after case completion] | ||
|
|
||
| **What we learned:** | ||
|
|
||
| Key insights gained: | ||
| Assumptions validated/invalidated: | ||
| Unexpected discoveries: | ||
| - Key insights gained: | ||
| - Assumptions validated/invalidated: | ||
| - Unexpected discoveries: | ||
|
|
||
| **What we would do differently:** | ||
| - Process improvements: | ||
| - Technical approach changes: | ||
|
|
||
| ## Review & Acknowledgment | ||
|
|
||
| [People who should review and acknowledge understanding of this experiment] | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. the case author and assigned people |
||
|
|
||
| Process improvements: | ||
| Technical approach changes: | ||
| - [ ] [Person 1] | ||
| - [ ] [Person 2] | ||
| - [ ] [Person 3] | ||
|
|
||
| *Note: Check your name after reading and understanding this case to confirm awareness and reduce communication overhead.* | ||
|
|
||
| --- | ||
|
|
||
| [ | ||
| **Final Checklist Before Submitting:** | ||
| - [ ] Does this describe Agent value, not technical implementation? | ||
| - [ ] Are agents prioritized with clear percentages and Human / System distinction? | ||
| - [ ] Is the Target System clearly identified and distinguished from consumer / dependency systems? | ||
| - [ ] Are system boundaries clearly defined? | ||
| - [ ] Is the language simple enough for non-native speakers? | ||
| - [ ] Is the scope limited to 3-6 months of achievable work? | ||
| - [ ] Do scenarios focus on agent behavior, not system performance? | ||
| ] | ||
| - [ ] Does this describe agent value, not technical implementation? | ||
| - [ ] Is problem analysis referenced (not duplicated) from Case Product Specification? | ||
| - [ ] Is Agent Priority Overview high-level with justification? | ||
| - [ ] Are acceptance criteria clear and verifiable? | ||
| - [ ] Do scenarios use correct terminology (Primary/Secondary/Alternative/Error)? | ||
| - [ ] Is scope limited to 3-6 months of achievable work? | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. more relevant to experiment |
||
| - [ ] Are only critical dependencies and blockers listed? | ||
| - [ ] Are links to Case Product Specification and Architecture docs present? | ||
| - [ ] Are experiment links marked as TBD where not yet planned? | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. not needed |
||
| - [ ] Is Review & Acknowledgment section complete? | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. not needed |
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.