Add literal() function to preserve apostrophes in shell arguments

.changeset/literal-apostrophe-fix.md

@@ -0,0 +1,22 @@
+---
+'command-stream': minor
+---
+
+Add `literal()` function to preserve apostrophes in shell arguments
+
+When passing text containing apostrophes to programs that store it literally (like API calls via CLI tools), apostrophes would appear corrupted as triple quotes (`'''`). The new `literal()` function uses double-quote escaping which preserves apostrophes while still escaping shell-dangerous characters.
+
+**New features:**
+
+- `literal(value)` - Mark text for double-quote escaping, preserving apostrophes
+- `quoteLiteral(value)` - Low-level function for manual command building
+
+**Usage:**
+
+```javascript
+import { $, literal } from 'command-stream';
+
+// Apostrophes now preserved for API storage
+const notes = "Dependencies didn't exist";
+await $`gh release create --notes ${literal(notes)}`;
+```

README.md

@@ -254,6 +254,60 @@ await $`echo ${userInput}`; // ✅ Safe - auto-escaped
 - ❌ Any untrusted source
 - ❌ When you're unsure - use normal interpolation instead
 
+### Preserving Apostrophes with `literal()` (Advanced)
+
+When passing text to programs that store it literally (like API calls via CLI tools), apostrophes can appear corrupted as triple quotes (`'''`). This happens because the default `quote()` function uses Bash's `'\''` escaping pattern for apostrophes.
+
+Use `literal()` to preserve apostrophes when the receiving program stores text literally:
+
+```javascript
+import { $, literal } from 'command-stream';
+
+// Problem: Default escaping uses '\'' pattern for apostrophes
+const releaseNotes = "Fix bug when dependencies didn't exist";
+await $`gh release create v1.0.0 --notes ${releaseNotes}`;
+// Apostrophe may appear as ''' in GitHub if the API stores it literally
+
+// Solution: Use literal() to preserve apostrophes
+await $`gh release create v1.0.0 --notes ${literal(releaseNotes)}`;
+// Apostrophe stays as ' - text displays correctly on GitHub
+
+// literal() still escapes shell-dangerous characters
+const text = "User's input with $variable and `backticks`";
+await $`command ${literal(text)}`;
+// $ and ` are escaped, but apostrophe stays as-is
+```
+
+**How `literal()` differs from `raw()` and default quoting:**
+
+| Function    | Apostrophe `'` | Shell Chars `$ \` "` | Safety Level |
+| ----------- | -------------- | -------------------- | ------------ |
+| Default     | `'\''` escaped | Safely quoted        | ✅ Maximum   |
+| `literal()` | Preserved      | Escaped              | ✅ Safe      |
+| `raw()`     | Preserved      | NOT escaped          | ⚠️ Dangerous |
+
+**When to use `literal()`:**
+
+- ✅ Text for APIs via CLI tools (GitHub, cloud CLIs)
+- ✅ Release notes, commit messages, descriptions
+- ✅ Any text that will be stored/displayed literally
+- ✅ When apostrophes appear as `'''` in the output
+
+**Recommended Alternative: Use stdin for APIs**
+
+For API calls, the safest approach is to pass data via stdin:
+
+```javascript
+const payload = JSON.stringify({
+  tag_name: 'v1.0.0',
+  body: releaseNotes, // No escaping issues!
+});
+
+await $`gh api repos/owner/repo/releases -X POST --input -`.run({
+  stdin: payload,
+});
+```
+
 ## Usage Patterns
 
 ### Classic Await (Backward Compatible)
@@ -1115,6 +1169,36 @@ await $`${raw(trustedCommand)}`;
 // ⚠️ NEVER use with untrusted input - shell injection risk!
 ```
 
+#### literal() - Preserve Apostrophes for Literal Storage
+
+Use when passing text to programs that store it literally (APIs, databases). Preserves apostrophes while still escaping shell-dangerous characters.
+
+```javascript
+import { $, literal } from 'command-stream';
+
+// Apostrophes preserved for API storage
+const notes = "Dependencies didn't exist";
+await $`gh release create v1.0.0 --notes ${literal(notes)}`;
+// → Apostrophe displays correctly on GitHub
+
+// Still safe: $ ` \ " are escaped
+const text = "User's $variable";
+await $`command ${literal(text)}`;
+// → $ is escaped, apostrophe preserved
+```
+
+#### quoteLiteral() - Low-level Double-Quote Escaping
+
+Low-level function for manual command building. Uses double quotes, preserving apostrophes.
+
+```javascript
+import { quoteLiteral } from 'command-stream';
+
+quoteLiteral("didn't"); // → "didn't"
+quoteLiteral('say "hello"'); // → "say \"hello\""
+quoteLiteral('$100'); // → "\$100"
+```
+
 ### Built-in Commands
 
 18 cross-platform commands that work identically everywhere:

docs/case-studies/issue-141/README.md

@@ -0,0 +1,169 @@
+# Case Study: Apostrophe Over-Escaping (Issue #141)
+
+## Overview
+
+This case study documents the issue where apostrophes in text arguments are over-escaped when passed through command-stream, causing them to appear as triple quotes (`'''`) when the text is stored or displayed literally by receiving programs.
+
+## Problem Statement
+
+When passing text containing apostrophes through command-stream in double-quoted template literals, apostrophes are escaped using Bash's `'\''` pattern. When the receiving command (like `gh` CLI) passes this text to an API that stores it literally, the escape sequences appear as visible characters.
+
+### Example
+
+```javascript
+const releaseNotes = "Fix bug when dependencies didn't exist";
+await $`gh release create v1.0.0 --notes "${releaseNotes}"`;
+// GitHub receives: "Fix bug when dependencies didn'\''t exist"
+// GitHub displays: "Fix bug when dependencies didn'''t exist"
+```
+
+## Timeline of Investigation
+
+1. **Initial report**: Issue observed in production with GitHub release notes
+2. **Related issue**: First documented in test-anywhere repository (issue #135)
+3. **Workaround implemented**: Using `gh api` with JSON stdin instead of shell arguments
+4. **Root cause identified**: Double-quoting when users add quotes around interpolated values
+
+## Root Cause Analysis
+
+### The Escaping Mechanism
+
+The `quote()` function in command-stream (js/src/$.mjs:1056-1100) handles shell escaping:
+
+```javascript
+function quote(value) {
+  // ... null/array handling ...
+
+  // Default: wrap in single quotes, escape internal single quotes
+  return `'${value.replace(/'/g, "'\\''")}'`;
+}
+```
+
+For input `didn't`, this produces `'didn'\''t'`, which is the correct Bash escaping for a single quote inside a single-quoted string.
+
+### The Double-Quoting Issue
+
+When users write:
+
+```javascript
+await $`command "${text}"`;
+```
+
+The template literal contains `"` characters as static strings. The `buildShellCommand()` function then:
+
+1. Adds the static string `command "`
+2. Calls `quote(text)` which wraps in single quotes
+3. Adds the closing static string `"`
+
+Result: `command "'escaped'\''text'"`
+
+This creates double-quoting - the user's `"..."` plus the library's `'...'`.
+
+### Experimental Evidence
+
+```
+Test: Direct shell (baseline)
+Command: /tmp/show-args.sh "Text with apostrophe's"
+Result: [Text with apostrophe's] ✅
+
+Test: With user-provided quotes (the bug)
+Command: $`/tmp/show-args.sh "${testText}"`
+Result: ['Dependencies didn'\''t exist'] ❌
+
+Test: Without user quotes (correct usage)
+Command: $`/tmp/show-args.sh ${testText}`
+Result: [Dependencies didn't exist] ✅
+```
+
+### Why Triple Quotes Appear
+
+When a program receives `'didn'\''t'`:
+
+1. If it **interprets** as shell → expands to `didn't` ✅
+2. If it **stores literally** → keeps as `'didn'\''t'` which displays as `didn'''t` ❌
+
+The `gh` CLI passes arguments to GitHub's API, which stores them literally without shell interpretation.
+
+## Solutions
+
+### Solution 1: Correct Usage (No User Quotes)
+
+The simplest solution is to not add quotes around interpolated values:
+
+```javascript
+// ❌ Wrong - double quoting
+await $`gh release create --notes "${text}"`;
+
+// ✅ Correct - let command-stream quote
+await $`gh release create --notes ${text}`;
+```
+
+### Solution 2: literal() Function (Proposed)
+
+Add a `literal()` function for cases where text should not be shell-escaped:
+
+```javascript
+import { $, literal } from 'command-stream';
+
+// Mark text as literal - minimal escaping for argument boundary only
+await $`gh release create --notes ${literal(releaseNotes)}`;
+```
+
+This would:
+
+- Apply only the minimal escaping needed for argument boundaries
+- Not apply Bash-specific patterns like `'\''`
+- Be useful when the receiving program stores text literally
+
+### Solution 3: Use stdin with JSON (Recommended for APIs)
+
+For API calls, pass data via stdin:
+
+```javascript
+const payload = JSON.stringify({
+  tag_name: 'v1.0.0',
+  body: releaseNotes,
+});
+
+await $`gh api repos/owner/repo/releases -X POST --input -`.run({
+  stdin: payload,
+});
+```
+
+This completely bypasses shell escaping issues.
+
+## Implementation Decision
+
+Based on the issue suggestions and analysis:
+
+1. **Implement `literal()` function** - For marking text that should not be shell-escaped
+2. **Improve documentation** - Clarify when and how shell escaping occurs
+3. **Add examples** - Show correct usage patterns and workarounds
+
+## Related Issues and References
+
+- **Issue #141**: This issue - Apostrophe over-escaping
+- **Issue #45**: Automatic quote addition in interpolation
+- **Issue #49**: Complex shell commands with nested quotes
+- **test-anywhere #135**: First observed occurrence
+- **test-anywhere PR #136**: Workaround using stdin/JSON
+
+## Lessons Learned
+
+1. Shell escaping and literal text storage are incompatible
+2. Users should not add quotes around interpolated values
+3. For API calls, JSON/stdin is the safest approach
+4. Clear documentation and examples are essential
+
+## Test Cases
+
+A proper fix should handle:
+
+| Input                | Expected Output      |
+| -------------------- | -------------------- |
+| `didn't`             | `didn't`             |
+| `it's user's choice` | `it's user's choice` |
+| `text is "quoted"`   | `text is "quoted"`   |
+| `it's "great"`       | `it's "great"`       |
+| `` use `npm` ``      | `` use `npm` ``      |
+| `Line 1\nLine 2`     | `Line 1\nLine 2`     |

eslint.config.js

@@ -144,12 +144,14 @@ export default [
     },
   },
   {
-    // Example and debug files are more lenient
+    // Example, experiment, and debug files are more lenient
     files: [
       'examples/**/*.js',
       'examples/**/*.mjs',
       'js/examples/**/*.js',
       'js/examples/**/*.mjs',
+      'experiments/**/*.js',
+      'experiments/**/*.mjs',
       'claude-profiles.mjs',
     ],
     rules: {