|
| 1 | +# ELIZA Demo Debug Session - December 31, 2025 |
| 2 | + |
| 3 | +## Summary |
| 4 | + |
| 5 | +Systematically debugged and tested the ELIZA demo for GitHub Pages deployment. |
| 6 | + |
| 7 | +## Bug Fixed |
| 8 | + |
| 9 | +### Bidirectional Post-Substitution Conflict |
| 10 | + |
| 11 | +**Location**: `/Users/jmanning/llm-course/demos/01-eliza/js/pattern-matcher.js` |
| 12 | + |
| 13 | +**Problem**: The `applyPostSubstitutions()` function was iterating through substitutions sequentially, causing bidirectional pairs to conflict: |
| 14 | +- `"you" -> "me"` followed by `"me" -> "you"` would cancel each other out |
| 15 | +- Same issue with `"your"/"my"` and `"yourself"/"myself"` |
| 16 | + |
| 17 | +**Solution**: Use placeholder markers during substitution: |
| 18 | +1. First pass: Replace all matches with unique placeholders (`__POSTSUB_0__`, `__POSTSUB_1__`, etc.) |
| 19 | +2. Second pass: Replace all placeholders with their final values |
| 20 | + |
| 21 | +This ensures atomic substitution without conflicts. |
| 22 | + |
| 23 | +## Test Suite |
| 24 | + |
| 25 | +Created comprehensive test suite at: |
| 26 | +`/Users/jmanning/llm-course/demos/01-eliza/test/eliza-tests.mjs` |
| 27 | + |
| 28 | +Run with: `node test/eliza-tests.mjs` |
| 29 | + |
| 30 | +**128 tests covering:** |
| 31 | +- Pre-substitutions (13 tests) |
| 32 | +- Post-substitutions/reflection (9 tests) |
| 33 | +- Pattern matching with wildcards (19 tests) |
| 34 | +- Capture groups (4 tests) |
| 35 | +- Rule matching by keyword (20 tests) |
| 36 | +- Response assembly (3 tests) |
| 37 | +- Edge cases (6 tests) |
| 38 | +- Synonym groups (29 tests) |
| 39 | +- Goto rule resolution (22 tests) |
| 40 | +- Rank priority (3 tests) |
| 41 | + |
| 42 | +## Commit |
| 43 | + |
| 44 | +`0637a4f` - fix(eliza): Fix bidirectional post-substitution conflict |
| 45 | + |
| 46 | +## Files Modified |
| 47 | + |
| 48 | +- `demos/01-eliza/js/pattern-matcher.js` - Fixed `applyPostSubstitutions()` function |
| 49 | +- `demos/01-eliza/test/eliza-tests.mjs` - Added comprehensive test suite |
| 50 | + |
| 51 | +## GitHub Issue |
| 52 | + |
| 53 | +Updated issue #26 with details: https://github.com/ContextLab/llm-course/issues/26#issuecomment-3703071348 |
| 54 | + |
| 55 | +## Notes for Future Work |
| 56 | + |
| 57 | +1. The "you" keyword has no explicit rank, while "my" has rank 2. This means sentences like "you remind me of my father" match "my" instead of the more specific "you remind me of" pattern. This is correct behavior per the ranking system but could be reconsidered. |
| 58 | + |
| 59 | +2. Pre-substitutions must be applied before `findMatchingRule()` for correct keyword detection (e.g., "computers" -> "computer"). |
| 60 | + |
| 61 | +3. The test suite uses ES modules (`.mjs` extension) and imports directly from the source files. |
| 62 | + |
| 63 | +--- |
| 64 | + |
| 65 | +## Python Parity Verification Session |
| 66 | + |
| 67 | +### Commit |
| 68 | + |
| 69 | +a2aa0dc - test(eliza): Fix keyword ranking test to match Python solution behavior |
| 70 | + |
| 71 | +### Analysis |
| 72 | + |
| 73 | +1. Verified punctuation handling matches Python |
| 74 | +2. Fixed keyword ranking test ("i" vs "always" - both rank 0, "i" appears first) |
| 75 | +3. All 128 tests passing |
| 76 | + |
| 77 | +### Issue #26 |
| 78 | + |
| 79 | +Added comment: https://github.com/ContextLab/llm-course/issues/26#issuecomment-3703166493 |
0 commit comments