LLM Whip

A TypeScript CLI tool that monitors code for lazy patterns and anti-cheat detection when working with LLMs.

Overview

LLM Whip detects common shortcuts and anti-patterns in code:

• TODO comments and placeholders
• Stub implementations
• "The important thing is..." statements
• Not implemented errors
• Other lazy coding patterns

Features

• Directory Auditing: Scan codebases for existing patterns
• Real-time Monitoring: Watch files as they change
• TypeScript Configuration: Type-safe configuration files
• Multiple Output Formats: Table, JSON, CSV export
• Baseline Tracking: Alert only on new patterns
• Configurable Patterns: Define custom detection rules

Quick Start

# Install
npm install -g llm-whip

# Monitor current directory with alerts
llm-whip

# Monitor with keyboard interrupts + sound
llm-whip --interrupt --sound

# Audit existing code
llm-whip audit ./src

Installation

bun add -g llm-whip
# or
npm install -g llm-whip

Usage

# Show help
llm-whip --help

# Monitor current directory
llm-whip

# Monitor specific directories
llm-whip ./src ./lib

# Monitor with keyboard interrupts (sends ESC + text + Enter to active window)
llm-whip ./src --interrupt

# Monitor with sound alerts
llm-whip ./src --sound

# Create configuration file
llm-whip init

# Audit current directory
llm-whip audit

Commands

Command	Description
llm-whip	Monitor current directory
llm-whip init [dir]	Create configuration file
llm-whip audit [dirs...]	Scan directories for patterns
llm-whip watch <dirs...>	Monitor directories in real-time

Options

Option	Description
--config=<path>	Custom configuration file
--format=<type>	Audit output format (table/json/csv)
--grep=<patterns>	Filter files by content patterns
--interrupt	Enable keyboard interrupts (sends ESC + text + Enter to active window)
--sound	Enable sound alerts

Configuration

LLM Whip supports both TypeScript (.ts) and JSON (.json) configuration files.

TypeScript Configuration

Create llm-whip.config.ts:

import type { Config } from 'llm-whip/types';

export const config: Config = {
	patterns: [
		{
			name: 'todo',
			pattern: 'TODO',
			severity: 'high',
			reactions: ['sound', 'alert', 'interrupt'],
			message: 'TODO comment detected',
			interruptMessage:
				'TODO comments should be completed before submitting code. Please implement the actual functionality instead of leaving placeholder comments.',
		},
		{
			name: 'important-thing',
			pattern: 'The important thing is',
			severity: 'medium',
			reactions: ['alert'],
			interruptMessage:
				"Detected 'The important thing is...' - this often indicates avoiding detailed implementation. Please provide specific, actionable details.",
		},
	],
	reactions: {
		sound: { command: 'afplay /System/Library/Sounds/Glass.aiff' },
		interrupt: {
			delay: 500,
			sequence: ['\\u001b', '{message}', '\\n'], // ESC + message + Enter
		},
		alert: { format: 'color' },
	},
	debounce: 2000,
	fileTracking: true,
};

JSON Configuration

Create llm-whip.config.json:

{
  "$schema": "https://raw.githubusercontent.com/bewinxed/llm-whip/main/schema.json",
  "patterns": [
    {
      "name": "todo",
      "pattern": "TODO",
      "severity": "high",
      "reactions": ["sound", "alert", "interrupt"],
      "message": "TODO comment detected",
      "interruptMessage": "TODO comments should be completed before submitting code."
    }
  ],
  "reactions": {
    "sound": true,
    "alert": { "format": "color" },
    "interrupt": {
      "delay": 500,
      "sequence": ["\\u001b", "{message}", "\\n"]
    }
  }
}

Default Patterns

Patterns use JavaScript regex syntax and are case-insensitive by default:

Pattern	Regex	Example Match
todo	TODO	// TODO: implement this
placeholder	placeholder|stub	// placeholder implementation
not-implemented	not implemented|NotImplementedError	throw new Error("not implemented")
important-thing	The important thing is	The important thing is to...

Custom Pattern Examples:

{
  name: "fixme",
  pattern: "FIXME\|BUG\|HACK",  // Matches FIXME, BUG, or HACK
  severity: "high"
},
{
  name: "console-log",
  pattern: "console\\.(log\|debug)",  // Matches console.log or console.debug
  severity: "low"
}

Keyboard Interrupts

The --interrupt flag enables keyboard interrupts that send detailed warnings to the active window when patterns are detected:

# Enable keyboard interrupts
llm-whip ./src --interrupt

When a pattern is detected, LLM Whip will:

1. Press ESC to clear any current input
2. Type a warning message to the active window
3. Press Enter to send the message
4. The message includes:
   • Pattern type and custom message
   • File path and line number
   • Timestamp

Note: The --interrupt flag will automatically add keyboard interrupts to all patterns, even if they don't have "interrupt" in their reactions array in the config file.

Each pattern can have a custom interruptMessage that gets sent:

{
  name: "todo",
  pattern: "TODO",
  reactions: ["interrupt"],
  interruptMessage: "TODO comments should be completed before submitting code. Please implement the actual functionality instead of leaving placeholder comments."
}

Sound Alerts

The --sound flag enables cross-platform sound alerts:

# Enable sound alerts
llm-whip ./src --sound

Default sounds by platform:

• macOS: Glass.aiff (fallback: Ping.aiff)
• Windows: Windows Critical Stop.wav (fallback: console beep)
• Linux: alarm-clock-elapsed.oga (fallback: bell.oga)

Monitoring LLM Conversations

LLM Whip can monitor LLM conversation outputs by watching log files. This helps detect anti-cheat patterns in both your code and the LLM's responses.

Method 1: Using tee to clone output

When using Claude Code CLI or other LLM tools, pipe the output to a file that LLM Whip monitors:

# Terminal 1: Start LLM Whip monitoring
llm-whip ./project ./logs

# Terminal 2: Run Claude Code with output logging
claude-code ./project 2>&1 | tee logs/claude-session.log

Method 2: Direct file monitoring

Create a log file and have your LLM tool write to it:

# Start monitoring the logs directory
llm-whip ./src ./logs

# Your LLM tool outputs to logs/conversation.txt
# LLM Whip will detect patterns in real-time

Method 3: Script wrapper

Create a wrapper script that automatically logs and monitors:

#!/bin/bash
# llm-monitor.sh

# Create logs directory
mkdir -p ./logs

# Start LLM Whip in background
llm-whip ./src ./logs &
WHIP_PID=$!

# Run your LLM tool with logging
claude-code "$@" 2>&1 | tee logs/session-$(date +%Y%m%d-%H%M%S).log

# Clean up
kill $WHIP_PID 2>/dev/null

Then use it like: ./llm-monitor.sh ./my-project

Advanced Usage

Baseline Tracking

Only alert on new patterns:

llm-whip audit ./src > /dev/null
llm-whip watch ./src

Export Formats

# JSON export
llm-whip audit ./src --format=json > issues.json

# CSV export
llm-whip audit ./src --format=csv > issues.csv

Configuration Priority

1. Local llm-whip.config.ts (current directory)
2. Custom path via --config=path
3. Built-in defaults

Development

bun install
bun test
bun run build

License

MIT