Skip to content

Commit d55e840

Browse files
Ubuntuclaude
authored andcommitted
docs(discoveries): User testing validates mandatory testing requirement
Issue #1783 user testing found critical bug (SubIssue not hashable) that 110 passing unit tests missed. Validates USER_PREFERENCES.md requirement. Bug would've shipped to production without user testing. Fixed in PR #1784. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
1 parent a14275a commit d55e840

File tree

3 files changed

+252
-16
lines changed

3 files changed

+252
-16
lines changed

.claude/context/DISCOVERIES.md

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ This file documents non-obvious problems, solutions, and patterns discovered dur
88

99
### Recent (December 2025)
1010

11+
- [Mandatory User Testing Validates Its Own Value](#mandatory-user-testing-validates-value-2025-12-02)
1112
- [System Metadata vs User Content in Git Conflict Detection](#system-metadata-vs-user-content-git-conflict-2025-12-01)
1213

1314
### November 2025
@@ -1787,3 +1788,127 @@ Opus V2:
17871788
**Pattern Identified**: Validation checkpoints can backfire - use flow language instead of interruption language
17881789

17891790
**Lesson**: Always validate AI guidance changes empirically with ALL target models before deploying
1791+
1792+
---
1793+
1794+
## Mandatory User Testing Validates Its Own Value {#mandatory-user-testing-validates-value-2025-12-02}
1795+
1796+
**Date**: 2025-12-02
1797+
**Context**: Implementing Parallel Task Orchestrator (Issue #1783, PR #1784)
1798+
**Impact**: HIGH - Validates mandatory testing requirement, found production-blocking bug
1799+
1800+
### Problem
1801+
1802+
Unit tests can achieve high coverage (86%) and 100% pass rate while missing critical real-world bugs.
1803+
1804+
### Discovery
1805+
1806+
Mandatory user testing (USER_PREFERENCES.md requirement) caught a **production-blocking bug** that 110 passing unit tests missed:
1807+
1808+
**Bug**: `SubIssue` dataclass not hashable, but `OrchestrationConfig` uses `set()` for deduplication
1809+
1810+
```python
1811+
# This passed all unit tests but fails in real usage:
1812+
config = OrchestrationConfig(sub_issues=[...])
1813+
# TypeError: unhashable type: 'SubIssue'
1814+
```
1815+
1816+
### How It Was Missed
1817+
1818+
**Unit Tests** (110/110 passing):
1819+
- Mocked all `SubIssue` creation
1820+
- Never tested real deduplication path
1821+
- Assumed API worked without instantiation
1822+
1823+
**User Testing** (mandatory requirement):
1824+
- Tried actual config creation
1825+
- **Bug discovered in <2 minutes**
1826+
- Immediate TypeError on first real use
1827+
1828+
### Fix
1829+
1830+
```python
1831+
# Before
1832+
@dataclass
1833+
class SubIssue:
1834+
labels: List[str] = field(default_factory=list)
1835+
1836+
# After
1837+
@dataclass(frozen=True)
1838+
class SubIssue:
1839+
labels: tuple = field(default_factory=tuple)
1840+
```
1841+
1842+
### Validation
1843+
1844+
**Test Results After Fix**:
1845+
```
1846+
✅ Config creation works
1847+
✅ Deduplication works (3 items → 2 unique)
1848+
✅ Orchestrator instantiation works
1849+
✅ Status API functional
1850+
```
1851+
1852+
### Key Insights
1853+
1854+
1. **High test coverage ≠ Real-world readiness**
1855+
- 86% coverage, 110/110 tests, still had production blocker
1856+
- Mocks hide integration issues
1857+
1858+
2. **User testing finds different bugs**
1859+
- Unit tests validate component logic
1860+
- User tests validate actual workflows
1861+
- Both are necessary
1862+
1863+
3. **Mandatory requirement justified**
1864+
- Without user testing, would've shipped broken code
1865+
- CI wouldn't catch this (unit tests pass)
1866+
- First user would've hit TypeError
1867+
1868+
4. **Time investment worthwhile**
1869+
- <5 minutes of user testing
1870+
- Found bug that could've cost hours of debugging
1871+
- Prevented embarrassing production failure
1872+
1873+
### Implementation
1874+
1875+
**Mandatory User Testing Pattern**:
1876+
```bash
1877+
# Test like a user would
1878+
python -c "from module import Class; obj = Class(...)" # Real instantiation
1879+
config = RealConfig(real_data) # No mocks
1880+
result = api.actual_method() # Real workflow
1881+
```
1882+
1883+
**NOT sufficient**:
1884+
```python
1885+
# Unit test approach (can miss real issues)
1886+
@patch("module.Class")
1887+
def test_with_mock(mock_class): # Never tests real instantiation
1888+
...
1889+
```
1890+
1891+
### Lessons Learned
1892+
1893+
1. **Always test like a user** - No mocks, real instantiation, actual workflows
1894+
2. **High coverage isn't enough** - Need real usage validation
1895+
3. **Mocks hide bugs** - Integration issues invisible to mocked tests
1896+
4. **User requirements are wise** - This explicit requirement saved us from shipping broken code
1897+
1898+
### Related
1899+
1900+
- Issue #1783: Parallel Task Orchestrator
1901+
- PR #1784: Implementation
1902+
- USER_PREFERENCES.md: Mandatory E2E testing requirement
1903+
- Commit dc90b350: Hashability fix
1904+
1905+
### Recommendation
1906+
1907+
**ENFORCE mandatory user testing** for ALL features:
1908+
- Test with `uvx --from git+...` (no local state)
1909+
- Try actual user workflows (no mocks)
1910+
- Verify error messages and UX
1911+
- Document test results in PR
1912+
1913+
This discovery **validates the user's explicit requirement** - mandatory user testing prevents production failures that unit tests miss.
1914+

docs/howto/string-utilities.md

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
# How to Use String Utilities
2+
3+
This guide shows you how to use the string utility functions in amplihack.
4+
5+
## Slugify Function
6+
7+
The `slugify()` function converts any string into a URL-safe slug.
8+
9+
### Basic Usage
10+
11+
```python
12+
from amplihack.utils.string_utils import slugify
13+
14+
# Convert text to slug
15+
result = slugify("Hello World")
16+
# Returns: "hello-world"
17+
```
18+
19+
### What It Does
20+
21+
The slugify function transforms your text by:
22+
23+
1. Normalizing unicode characters (accents become plain letters)
24+
2. Converting to lowercase
25+
3. Removing special characters
26+
4. Replacing spaces and underscores with hyphens
27+
5. Collapsing multiple hyphens into one
28+
6. Stripping leading and trailing hyphens
29+
30+
### Common Examples
31+
32+
**Basic text conversion:**
33+
34+
```python
35+
slugify("My Blog Post Title") # Returns: "my-blog-post-title"
36+
```
37+
38+
**Handling special characters:**
39+
40+
```python
41+
slugify("Hello@World!") # Returns: "hello-world"
42+
slugify("Rock & Roll") # Returns: "rock-roll"
43+
```
44+
45+
**Unicode normalization:**
46+
47+
```python
48+
slugify("Café") # Returns: "cafe"
49+
slugify("Crème brûlée") # Returns: "creme-brulee"
50+
```
51+
52+
**Numbers are preserved:**
53+
54+
```python
55+
slugify("Project 123 Version 2") # Returns: "project-123-version-2"
56+
slugify("test123") # Returns: "test123"
57+
```
58+
59+
**Underscores converted to hyphens:**
60+
61+
```python
62+
slugify("hello_world") # Returns: "hello-world"
63+
```
64+
65+
### Edge Cases
66+
67+
**Empty strings:**
68+
69+
```python
70+
slugify("") # Returns: ""
71+
slugify(" ") # Returns: ""
72+
```
73+
74+
**Only special characters:**
75+
76+
```python
77+
slugify("!!!") # Returns: ""
78+
slugify("@#$") # Returns: ""
79+
```
80+
81+
**Already valid slugs:**
82+
83+
```python
84+
slugify("already-a-slug") # Returns: "already-a-slug"
85+
```
86+
87+
**Emoji and unicode:**
88+
89+
```python
90+
slugify("Hello 😀 World") # Returns: "hello-world"
91+
```
92+
93+
### Return Value
94+
95+
The function always returns a string:
96+
97+
- Valid slugs contain only lowercase letters, numbers, and single hyphens
98+
- No leading or trailing hyphens
99+
- No consecutive hyphens
100+
- Returns empty string if input contains no valid characters
101+
102+
### Function Signature
103+
104+
```python
105+
def slugify(text: str) -> str:
106+
"""Convert string to URL-safe slug.
107+
108+
Args:
109+
text: String to convert
110+
111+
Returns:
112+
URL-safe slug string (lowercase, alphanumeric + hyphens)
113+
"""
114+
```
115+
116+
### Idempotency
117+
118+
Slugify is idempotent - running it twice gives the same result:
119+
120+
```python
121+
original = "Hello World!"
122+
first = slugify(original) # "hello-world"
123+
second = slugify(first) # "hello-world" (same)
124+
```
125+
126+
This means it's safe to slugify text that might already be a slug.

tests/unit/test_string_utils.py

Lines changed: 1 addition & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -26,22 +26,7 @@
2626
# Add src to path for imports
2727
sys.path.insert(0, str(Path(__file__).parent.parent.parent / "src"))
2828

29-
30-
# slugify function to be implemented
31-
try:
32-
from amplihack.utils.string_utils import slugify
33-
except ImportError:
34-
# Define placeholder so tests can be written
35-
def slugify(text: str) -> str:
36-
"""Placeholder - to be implemented.
37-
38-
Args:
39-
text: String to convert to slug
40-
41-
Returns:
42-
URL-safe slug string
43-
"""
44-
raise NotImplementedError("slugify not yet implemented")
29+
from amplihack.utils.string_utils import slugify
4530

4631

4732
class TestSlugify:

0 commit comments

Comments
 (0)