LazyTail Roadmap

This is a local planning document for upcoming features and improvements.

Current Status (v0.6.0)

Core Features Complete:

Lazy file reading with indexed line positions
Live filtering with background processing
File watching and auto-reload
Follow mode (tail -f style)
Filter history with arrow key navigation
ANSI color support
Vim-style line jumping (:123)
Vim-style z commands (zz, zt, zb)
Mouse scroll support
Help overlay (? key)
Event-based architecture

v0.2.0 Features:

Multi-tab support with side panel UI
Stdin support (cmd | lazytail)
Multiple file arguments (lazytail a.log b.log)
Per-tab state (filter, scroll, follow mode)
Tab navigation (Tab, Shift+Tab, 1-9)
AUR package available

v0.3.0 Features:

Regex filter mode (Tab to toggle)
Case sensitivity toggle (Alt+C)
Filter history with mode persistence
Expandable log entries (Space to toggle, c to collapse)
Persistent filter history to disk
Stats panel (line counts)
Filter progress percentage display
Streaming filter with SIMD search (memmem) for better performance
Grep-style search for case-sensitive patterns

v0.4.0 Features:

Source discovery mode (lazytail with no args)
Source capture mode (cmd | lazytail -n "Name")
Active/ended status indicators for discovered sources
Directory watcher for dynamic tab creation
Close tab with confirmation dialog (x / Ctrl+W)
MCP server support (lazytail --mcp)
MCP tools: list_sources, get_lines, get_tail, search, get_context
Streaming filter optimization for MCP (grep-like performance on 5GB+ files)

v0.5.0 Features:

Config system with lazytail.yaml discovery (walk parent directories)
lazytail init and lazytail config {validate,show} subcommands
Project-scoped and global source definitions in config
Query language: json | field == "value" syntax in filter input
MCP query language integration (JSON and text syntax converge on shared AST)
MCP plain text output format (default) to reduce JSON escaping overhead
Display file path in header and y to copy source path
Project-local data directories (.lazytail/)

v0.6.0 Features:

Columnar index system with severity detection
Index-accelerated filtering with bitmap pre-filtering
Severity-based line coloring (ERROR/WARN/INFO/DEBUG)
Severity histogram in stats panel
Line count and file size per source in side panel
MCP get_stats tool with index metadata and severity breakdown
Incremental index building during capture mode
O(1) line access via mmap-backed columnar offsets

v0.7.0 Features:

Self-update (lazytail update) with package manager detection

Post-v0.7.0 (unreleased):

Scrollable help overlay with j/k navigation
MCP get_tail since_line parameter for incremental polling
Copy selected line to clipboard (y) with OSC 52, ANSI stripping, status bar feedback
Mouse click: click sources in side panel to switch tabs, click log lines to select, click category headers to expand/collapse

Upcoming Features & Improvements

🔴 HIGH PRIORITY

Phase 1: Multi-Tab Support (CLI Arguments) ✅

Goal: View multiple log files in tabs within single UI instance

Status: Complete (v0.2.0)

lazytail api.log worker.log db.log
# Opens UI with side panel showing all sources

UI Layout:

┌──────────────┬──────────────────────────────────────────────────────┐
│ Sources      │ [log content]                                        │
│──────────────│                                                      │
│ > api.log    │ 10:00:01 INFO  Starting server...                   │
│   worker.log │ 10:00:02 DEBUG Connected to DB                      │
│   db.log     │ 10:00:03 INFO  Listening on :8080                   │
│              │ 10:00:04 ERROR Connection refused                   │
│──────────────│ 10:00:05 INFO  GET /health 200                      │
│ Severity     │                                                      │
│──────────────│                                                      │
│ ○ FATAL    0 │                                                      │
│ ● ERROR   12 │                                                      │
│ ○ WARN    45 │                                                      │
│ ○ INFO   892 │                                                      │
│ ○ DEBUG   45 │                                                      │
│──────────────│──────────────────────────────────────────────────────│
│ [Bookmarks]  │ Filter: _                    Showing 12/1183 ⟳ 45%   │
└──────────────┴──────────────────────────────────────────────────────┘

Status bar (right-aligned indicators):
- "Showing X/Y" - filtered count / total count
- "⟳ 45%" - filter processing progress (hidden when idle)
- "●" - follow mode active indicator

Two-panel layout:
- Left:   Source list, severity filter, bookmarks (future)
- Right:  Log content + filter input

Side Panel Design:

Left panel shows all available sources
Tree structure ready for future organization (folders, groups)
Active source highlighted with >
Shows indicators: * for unsaved filter, ● for active/live source
Panel can be toggled hidden/visible (e.g., Ctrl+B)
Future: Bookmarks section at bottom for project-scoped quick access

Tasks:

Future Side Panel Enhancements:

Show total line count and file size per source in side panel (live-updating as file grows) — ✅ v0.6.0
Fix: selected empty/ended source is invisible — grayed-out dim text has no visible selection highlight, making it unclear which tab is active
Tree structure with collapsible groups
Drag-and-drop reordering
Bookmarks section (per UI instance / project scope)
- Save frequently used file combinations
- Quick switch between "projects"
- Persist bookmarks to config file
Search/filter within source list

Use Cases:

# Compare multiple services
lazytail api.log worker.log scheduler.log

# System logs
lazytail /var/log/syslog /var/log/auth.log

# Multiple container logs (pre-captured)
lazytail pod1.log pod2.log pod3.log

Phase 2: Source Discovery ✅

Goal: Auto-discover log sources from config directory

Status: Complete

lazytail              # No args → discover sources from ~/.config/lazytail/data/
lazytail api.log      # Explicit file → single tab (backward compatible)

Directory Structure:

~/.config/lazytail/
├── data/             # Log files (auto-discovered)
│   ├── API.log
│   ├── Worker.log
│   └── DB.log
└── sources/          # Active source markers
    ├── API           # Contains PID, indicates source is live
    └── Worker

Tasks:

Behavior:

lazytail (no args) → discover mode, show all sources from config dir
lazytail file.log → explicit mode, show only that file
lazytail file1.log file2.log → explicit mode, show those files

Phase 3: Source Capture Mode (Tee-like) ✅

Goal: Capture stdin to named source, viewable in UI

Status: Complete

# Capture logs from any command
cmd | lazytail -n "API"
lazytail -n "API" <(kubectl logs -f pod)

# Works like:
# cmd | tee ~/.config/lazytail/data/API.log
# + register in sources/ + collision check + header

Tasks:

Full Workflow:

# Terminal 1: Capture API logs
kubectl logs -f api-pod | lazytail -n "API"

# Terminal 2: Capture worker logs
kubectl logs -f worker-pod | lazytail -n "Worker"

# Terminal 3: View everything
lazytail
# Shows tabs: [API] [Worker]
# API marked as "active", Worker marked as "active"

# Kill Terminal 1
# UI shows: API now marked as "ended", history still available

Future Enhancements (Post-Phase 3)

lazytail -n should truncate (reset) existing log file by default instead of appending
- Current behavior: appends to existing log file, accumulating stale data across runs
- New default: truncate the file on start so each capture session begins fresh
- Add --append / -a flag to preserve existing contents (opt-in)
Session ID for capture runs
- Each lazytail -n invocation generates a unique session ID (e.g., UUID or timestamp-based)
- Write a session boundary marker to the log file on start (e.g., --- session: abc123 started at 2026-02-23T10:00:00 ---)
- Store session ID in the marker file alongside PID
- Store session ID in index metadata or as a checkpoint annotation
- Expose in list_sources response so MCP consumers can see the current session
- Enable filtering by session: session == "abc123" or session == "latest" in query language
- Use case: when logs accumulate across multiple runs, users can filter to just the current run without clearing old data
- Pairs well with the truncate-on-start option above (session ID works when you want to keep history)
--file <path> for custom log file location
--max-size <size> for log rotation
Memory-only mode with streaming (no file)
Merged chronological view across sources
Filter across all tabs simultaneously

Phase 5: Query Language (LogQL-style) 🔴 HIGHEST PRIORITY

Goal: Unified pipeline-based query language for filtering, time ranges, and aggregation - with dual input formats (text for UI, JSON for MCP/LLMs)

Architecture:

┌─────────────────────┐      ┌─────────────────────┐
│  Text Query (UI)    │      │  JSON Query (MCP)   │
│                     │      │                     │
│ json | level=="err" │      │ {"parser":"json",   │
│                     │      │  "filters":[...]}   │
└──────────┬──────────┘      └──────────┬──────────┘
           │  parse                     │  deserialize
           ▼                            ▼
      ┌────────────────────────────────────┐
      │         FilterQuery (AST)          │
      └──────────────────┬─────────────────┘
                         │  execute
                         ▼
                  ┌─────────────┐
                  │   Results   │
                  └─────────────┘

Key Insight: MCP tool parameters ARE the query language for LLMs. Design rich structured JSON parameters that compile to the same AST as text queries.

Text Syntax (for humans):

# Field filtering
json | level == "error" | service =~ "api|worker"

# Exclusion (critical for noisy logs)
json | level == "error" | msg !~ "kscreen|systemd"

# Time filtering
json | time > "2024-01-28T10:00:00" | time < "2024-01-28T11:00:00"

# Aggregation
json | level == "error" | count by (service)
json | count by (level) | top 10

JSON Syntax (for MCP/LLMs):

{
  "parser": "json",
  "filters": [
    {"field": "level", "op": "==", "value": "error"},
    {"field": "service", "op": "=~", "value": "api|worker"}
  ],
  "exclude": [
    {"field": "msg", "pattern": "kscreen|systemd"}
  ],
  "time_range": {
    "field": "timestamp",
    "after": "2024-01-28T10:00:00",
    "before": "2024-01-28T11:00:00"
  },
  "aggregate": {
    "count_by": "service",
    "limit": 10
  }
}

Pipeline Stages:

Stage	Text Syntax	JSON Field	Description
Parser	`json`, `logfmt`, `pattern "..."`	`parser`	Extract fields from line
Filter	`field == "value"`	`filters[]`	Include matching lines
Exclude	`field !~ "pattern"`	`exclude[]`	Remove matching lines
Time	`time > "..."`	`time_range`	Filter by timestamp
Aggregate	`count by (field)`	`aggregate`	Group and count
Limit	`top N`	`aggregate.limit`	Limit results

Operators:

Operator	Description	Example
`==`, `!=`	Equality	`level == "error"`
`=~`, `!~`	Regex match/exclude	`msg !~ "kscreen"`
`>`, `<`, `>=`, `<=`	Comparison (numeric/time)	`status >= 500`
`contains`	Substring match	`msg contains "timeout"`

FilterQuery AST (Rust):

struct FilterQuery {
    parser: Parser,                    // json, logfmt, pattern, raw
    filters: Vec<FieldFilter>,         // field op value
    exclude: Vec<ExcludePattern>,      // negative filters
    time_range: Option<TimeRange>,     // after/before timestamps
    aggregate: Option<Aggregation>,    // count_by, limit
}

enum Parser {
    Raw,                               // plain text (default)
    Json,                              // parse as JSON
    Logfmt,                            // parse key=value
    Pattern(String),                   // extract via pattern
}

struct FieldFilter {
    field: String,                     // e.g., "level" or "user.id"
    op: Operator,                      // ==, !=, =~, !~, >, <, etc.
    value: Value,                      // string, number, regex
}

struct Aggregation {
    count_by: Option<String>,          // group by field
    limit: Option<usize>,              // top N
}

Implementation Order (MCP-first):

Define AST structs with serde derives — ✅ v0.5.0
Build executor that processes FilterQuery — ✅ v0.5.0
JSON deserialization → MCP tools work immediately — ✅ v0.5.0
Text parser → UI gets query language later — ✅ v0.5.0

Tasks:

Phase 4: Advanced Filter Modes

Goal: Add regex filtering and case sensitivity with intuitive mode switching

UX Design:

┌─────────────────────────────────────────────────────────────┐
│ Plain text mode (default):                                  │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ Filter: error                              [Tab: Regex] │ │
│ └─────────────────────────────────────────────────────────┘ │
│ Frame color: default (e.g., white/gray)                     │
└─────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────┐
│ Regex mode:                                                 │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ Regex: error|warn|fatal                    [Tab: Plain] │ │
│ └─────────────────────────────────────────────────────────┘ │
│ Frame color: distinct (e.g., cyan/magenta)                  │
└─────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────┐
│ Invalid regex (visual feedback):                            │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ Regex: error[                              [Tab: Plain] │ │
│ └─────────────────────────────────────────────────────────┘ │
│ Frame color: red (indicates error)                          │
└─────────────────────────────────────────────────────────────┘

Behavior:

Tab while in filter input: toggles between plain text and regex mode
Filter panel frame color changes to indicate current mode
Invalid regex: frame turns red, filter not applied until valid
Reopening filter (/) restores last used mode
History stores mode per entry, navigating history switches mode automatically
Case sensitivity toggle available in both modes

Filter Mode States:

FilterMode {
    Plain { case_sensitive: bool },
    Regex { case_sensitive: bool },
}

History Entry:

FilterHistoryEntry {
    pattern: String,
    mode: FilterMode,
}

Keybindings (while in filter input):

Tab - Toggle between Plain/Regex mode
Ctrl+I - Toggle case sensitivity
Up/Down - Navigate history (mode switches automatically)
Enter - Apply filter
Esc - Cancel

Visual Indicators:

Mode	Frame Color	Label
Plain (case-insensitive)	Default	`Filter:`
Plain (case-sensitive)	Default	`Filter [Aa]:`
Regex (case-insensitive)	Cyan	`Regex:`
Regex (case-sensitive)	Cyan	`Regex [Aa]:`
Regex (invalid)	Red	`Regex:`

Tasks:

Current Status: ✅ Complete

Explicit Query Filter Mode

Goal: Add Query as a first-class filter mode alongside Plain and Regex, instead of auto-detecting query syntax

Problem: Query mode is currently triggered by heuristic (is_query_syntax) — if the input looks like json | ..., it's silently treated as a query. This means:

Searching for a literal string like json | something is impossible in Plain mode
The mode switch is invisible to the user (no explicit trigger)
No way to force query mode for edge-case inputs that don't pass the heuristic

Proposed UX:

Plain → Regex → Query  (Tab cycles through all three)

┌─────────────────────────────────────────────────────────────┐
│ Query mode:                                                  │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ Query: json | level == "error"          [Tab: Plain]    │ │
│ └─────────────────────────────────────────────────────────┘ │
│ Frame color: magenta (already used for query today)         │
└─────────────────────────────────────────────────────────────┘

Behavior:

Tab cycles: Plain → Regex → Query → Plain
In Plain and Regex modes, input is never interpreted as a query — json | something is a literal search
Query mode always routes through QueryFilter, no heuristic needed
Auto-detect (is_query_syntax) can be removed or kept only as an optional hint (e.g. auto-switch offer)
History entries store mode, so query history restores Query mode

Tasks:

Add Query variant to FilterMode enum
Update Tab key to cycle Plain → Regex → Query
Remove heuristic auto-detection from filter dispatch path
Update filter prompt label and frame color for Query mode
Update help text and help overlay
Update filter history serialization for new mode variant
Add tests for mode cycling and query mode dispatch

🟡 MEDIUM PRIORITY

Expandable Log Entries ✅

Goal: Open/expand log entries to view full content (long lines, JSON properties)

Status: Implemented - Space to toggle, 'c' to collapse all

Use Cases:

View truncated long lines in full
Pretty-print JSON log entries
Inspect multi-line stack traces
Copy full content of a log entry

UI Behavior:

Normal view (collapsed):
┌─────────────────────────────────────────────────────────┐
│ 142  2024-01-20 10:00:01 {"level":"error","msg":"Fai...│
│ 143  2024-01-20 10:00:02 Starting worker process       │
│ 144  2024-01-20 10:00:03 Connection established        │
└─────────────────────────────────────────────────────────┘

Expanded view (press Enter or 'o' on line 142):
┌─────────────────────────────────────────────────────────┐
│ 142  2024-01-20 10:00:01 {"level":"error","msg":"Fai...│
│ ┌─────────────────────────────────────────────────────┐ │
│ │ {                                                   │ │
│ │   "level": "error",                                 │ │
│ │   "msg": "Failed to connect to database",          │ │
│ │   "error": "connection refused",                   │ │
│ │   "host": "db.example.com",                        │ │
│ │   "port": 5432,                                    │ │
│ │   "retry_count": 3                                 │ │
│ │ }                                                   │ │
│ └─────────────────────────────────────────────────────┘ │
│ 143  2024-01-20 10:00:02 Starting worker process       │
│ 144  2024-01-20 10:00:03 Connection established        │
└─────────────────────────────────────────────────────────┘

Raw expanded view (for non-JSON long lines):
┌─────────────────────────────────────────────────────────┐
│ 142  2024-01-20 10:00:01 Very long log message that ...│
│ ┌─────────────────────────────────────────────────────┐ │
│ │ Very long log message that contains a lot of       │ │
│ │ information and spans multiple lines when fully    │ │
│ │ displayed without truncation so you can read the   │ │
│ │ entire content of the log entry.                   │ │
│ └─────────────────────────────────────────────────────┘ │
│ 143  2024-01-20 10:00:02 Starting worker process       │
└─────────────────────────────────────────────────────────┘

Tasks:

Display Modes (per entry):

Raw: Word-wrapped full text (default for non-JSON)
JSON: Pretty-printed with syntax highlighting
Auto: Detect format and choose appropriate mode

Future:

Collapsible JSON nodes (expand/collapse nested objects)
Table view for structured logs
Custom formatters for known log formats

Stats Panel (Left Column)

Goal: Show log statistics in the left panel below the source list

UI Layout:

┌──────────────────────┬─────────────────────────────────────────┐
│ Sources              │ [log content]                           │
│──────────────────────│                                         │
│ > api.log            │ 142  INFO  Starting server...          │
│   worker.log         │ 143  DEBUG Connected to database       │
│   db.log             │ 144  ERROR Failed to connect      ← red│
│                      │                                         │
│──────────────────────│                                         │
│ Stats                │                                         │
│──────────────────────│                                         │
│ Lines:      1,234    │                                         │
│ Filtered:     892    │                                         │
│                      │                                         │
│ ERROR          12    │                                         │
│ WARN           45    │                                         │
│ INFO          892    │                                         │
│ DEBUG         285    │                                         │
└──────────────────────┴─────────────────────────────────────────┘

Features:

Total line count and filtered count
Severity breakdown with counts (requires severity detection)
Updates in real-time as file changes or filter applied
Clickable severity levels to quick-filter (future)

Tasks:

Current Status: ✅ Complete (v0.6.0) — stats panel shows line counts and severity histogram with color-coded display

Log Format Detection & Severity Parsing ✅

Goal: Automatically detect log format and extract severity for highlighting and filtering

Status: ✅ Complete (v0.6.0) — columnar index system with byte-level severity detection

Severity Levels (standardized):

TRACE → DEBUG → INFO → WARN → ERROR → FATAL

Detection Sources:

Format	Example	Severity Extraction	Status
JSON	`{"level":"error","msg":"..."}`	Parse `level`, `severity`, `lvl` fields	✅ v0.6.0
Bracket	`[ERROR] Failed to connect`	Match `[LEVEL]` pattern	✅ v0.6.0
Prefix	`ERROR: Connection refused`	Match `LEVEL:` pattern	✅ v0.6.0
Syslog	`<3>Jan 20 10:00:01 app[123]: msg`	Parse priority code	✅ v0.6.0
Log4j	`2024-01-20 ERROR com.app - msg`	Match known patterns	✅ v0.6.0
Kubernetes	`E0120 10:00:01.123 file.go:42]`	First char: I/W/E/F	✅ v0.6.0

UI Integration (Left Panel):

┌──────────────┬──────────────────────────────────────────────────────┐
│ Sources      │ [log content]                                        │
│──────────────│                                                      │
│ > api.log    │ 142  INFO  Starting server...                       │
│   worker.log │ 143  DEBUG Connected to database                    │
│   db.log     │ 144  ERROR Failed to authenticate             ← red │
│              │ 145  WARN  Retry attempt 2/3                  ← yel │
│──────────────│ 146  INFO  Request processed                        │
│ Severity     │                                                      │
│──────────────│                                                      │
│ ○ FATAL    0 │                                                      │
│ ● ERROR   12 │ ← active filter                                      │
│ ○ WARN    45 │                                                      │
│ ○ INFO   892 │                                                      │
│ ○ DEBUG  234 │                                                      │
│──────────────│──────────────────────────────────────────────────────│
│ [Bookmarks]  │ Filter: database              Showing 12/1183 ⟳ 100% │
└──────────────┴──────────────────────────────────────────────────────┘

Severity Section Features:

Severity levels with counts (from current source)
Toggle filtering: click/select to show only that level and above
● indicates active filter, ○ indicates inactive
Counts update as text filter changes
Keybinding to cycle severity filter (e.g., s to cycle through levels)

Tasks:

Future:

Custom format definitions (regex-based) in config
Timestamp parsing from detected format
Auto-detect field names for structured logs
Columnar index with flags, offsets, checkpoints — ✅ v0.6.0
Index-accelerated filtering with bitmap pre-filtering — ✅ v0.6.0

Persist Filter History to Disk ✅

Goal: Save filter history between sessions

Tasks:

Add history file path (~/.config/lazytail/history.json)
Load history on startup
Save history after each filter submission
Handle file read/write errors gracefully
Add tests for persistence

Current Status: ✅ Complete

Benefits:

Persistent workflow across sessions
Better UX for repeated log analysis

🟢 LOW PRIORITY

Mouse Controls

Goal: Expand mouse support beyond scroll — make the UI fully mouse-interactive

Tasks:

Click to select a log line
Click source in side panel to switch tabs
Click severity levels in stats panel to filter (when severity stats land)
Click-and-drag to select text for copying
Right-click context menu (expand, copy, filter by selection)
Resize side panel by dragging the divider
Double-click to expand/collapse a log line

Benefits:

Lower barrier to entry for non-vim users
Faster interaction for common actions (tab switching, line selection)
Expected by users coming from GUI log viewers

Multi-Line Selection

Goal: Select a range of log lines for copying, exporting, or inspection

Behavior:

V enters visual/selection mode (vim-style), Esc exits
Shift+Up/Shift+Down or Shift+j/Shift+k to extend selection
Shift+Click to select range from current line to clicked line
Click-and-drag to select range with mouse
Selected lines highlighted with distinct background color
y copies all selected lines to clipboard
Esc or movement without Shift clears selection
Selection works with both filtered and unfiltered views

Tasks:

Selection state in viewport (anchor line + current line range)
Visual mode toggle (V) and keyboard range extension
Mouse selection (Shift+Click range, click-and-drag)
Render selected lines with highlight background
y copies selected range to clipboard (full raw content, newline-separated)
Selection count indicator in status bar ("3 lines selected")

Search Highlighting

Goal: Highlight filter matches in displayed text

Tasks:

Detect filter pattern in rendered lines
Apply highlight style to matching substrings
Handle case sensitivity in highlighting
Support regex pattern highlighting
Add tests with mock rendering
Make highlight colors configurable

Benefits:

Visual feedback for matches
Easier to spot relevant content
Common feature in log viewers

Scrollable Help Overlay ✅

Goal: Make the help overlay scrollable so all keybindings are visible on small terminals

Status: ✅ Complete (post-v0.7.0)

Tasks:

Track a scroll offset for the help overlay
Handle j/k and ↑/↓ to scroll while help is open
Show a scroll indicator (e.g. ↓ more at the bottom)
Clamp scroll so the last line stays visible

Structured Logging & Debug Instrumentation 🔴 HIGH PRIORITY

Goal: Add comprehensive logging to debug what LazyTail is doing internally

Motivation: Currently difficult to debug issues like:

Why is filtering slow on this file?
Which reader implementation is being used?
Why did the file watcher trigger?
What's happening during index builds?
Why is follow mode not working?
Performance bottlenecks in the event loop

Logging Framework:

Use tracing crate (better than log for structured context)
Support multiple output targets (stderr, file, structured JSON)
Configurable per-module log levels
Span-based instrumentation for performance tracing

Key Areas to Instrument:

File Operations
- Reader selection (FileReader vs HugeFileReader vs StreamReader)
- File watching events (what changed, how many bytes)
- Index building progress and timing
- Mmap operations and failures
Filtering
- Filter orchestrator decisions (which engine is used)
- Filter progress (lines scanned, matches found, elapsed time)
- Streaming filter vs generic filter selection
- Query parsing and execution
Event Loop
- Event types received and processing time
- Debouncing decisions
- Frame timing (render, collect, process)
- Dropped frames / performance issues
MCP Server
- Tool invocations (which tool, parameters)
- Query execution time
- Result sizes
- Errors and failures
Capture Mode
- Lines captured per second
- Flush events
- Signal handling
- Marker file operations

Log Levels:

ERROR: Failures that impact functionality
WARN: Degraded performance, recoverable errors
INFO: Major operations (file opened, filter applied, source added)
DEBUG: Detailed operation info (event types, state transitions)
TRACE: Verbose instrumentation (every line read, every event)

Configuration:

# Enable debug logs for filter module
RUST_LOG=lazytail::filter=debug lazytail app.log

# Enable trace for everything
RUST_LOG=trace lazytail app.log

# Log to file
RUST_LOG=debug lazytail app.log 2> debug.log

# Structured JSON output
RUST_LOG_FORMAT=json RUST_LOG=debug lazytail --mcp

Performance Tracing:

use tracing::{info_span, instrument};

#[instrument(skip(reader))]
fn apply_filter(reader: &dyn LogReader, filter: Arc<dyn Filter>) {
    let _span = info_span!("apply_filter", total_lines = reader.total_lines()).entered();
    // ... filtering logic
    // Automatically logs: duration, total_lines, function args
}

Tasks:

Benefits:

Debuggability: Understand what LazyTail is doing without recompiling
Performance analysis: Find bottlenecks with span timing
User support: Ask users for logs instead of guessing
Development: Faster iteration when debugging issues
Production monitoring: Track MCP server performance

Future Ideas (Backlog)

Performance & Scalability

Features

Project-Scoped Instances (lazytail.yaml) ✅

Goal: Per-project log sources and configuration, auto-discovered by ancestry

Status: Core config system implemented (v0.4.0)

Discovery Order:

Check current dir and ancestors for lazytail.yaml
If found → project mode (use .lazytail/ in that dir)
If not found → global mode (~/.config/lazytail/)

Directory Structure:

my-project/
├── lazytail.yaml          # Config (committed to git)
├── .lazytail/             # Data (gitignored)
│   ├── data/              # Captured logs
│   ├── sources/           # Active markers
│   └── history.json       # Project-specific filter history
└── src/

lazytail.yaml Example:

# Source definitions (path-based only)
sources:
  - name: Database
    path: /var/log/postgresql/postgresql.log
  - name: App
    path: ./logs/app.log  # Relative to project root
  - name: Nginx
    path: /var/log/nginx/access.log

Benefits:

Team shares source definitions via git
AI assistants (Claude Code) auto-discover project logs
No pollution of global config
Project-specific filter history
Different projects can have different log setups

Tasks:

Config file discovery (walk ancestors for lazytail.yaml)
Parse YAML config with serde
Create .lazytail/ directory structure
Support path: sources (watch existing file)
Relative path resolution from project root
Filter presets in config
MCP: detect project root and scope sources
Fallback to global ~/.config/lazytail/ when no project found

Developer Experience

Integration tests for full app behavior
UI snapshot testing
Performance benchmarks in CI
Release automation improvements — ✅ 2026-02-20
- Auto-trigger release builds when release-please creates releases
- Binaries automatically attached to GitHub releases
Pre-built binaries for Windows

MCP Tools Roadmap

Current Tools (v0.6.0):

Tool	Purpose	Status
`list_sources`	Discover available log sources	✅ Complete
`get_lines`	Read lines from position	✅ Complete
`get_tail`	Read last N lines	✅ Complete
`search`	Find pattern matches + structured queries	✅ Complete
`get_context`	Get lines around a match	✅ Complete
`get_stats`	Index metadata and severity breakdown	✅ Complete (v0.6.0)

Common Parameters (all tools except list_sources):

Parameter	Type	Default	Description
`output`	text/json	text	Response format (text reduces escaping for AI)
`raw`	bool	false	Keep ANSI escape codes (default strips them)

Current search Parameters:

Parameter	Type	Status
`source`	String	✅ Done
`pattern`	String	✅ Done (optional when using `query`)
`mode`	plain/regex	✅ Done
`case_sensitive`	bool	✅ Done
`max_results`	usize	✅ Done
`context_lines`	usize	✅ Done
`query`	FilterQuery	✅ Done (JSON/logfmt field filtering with exclusions)
`time_range`	TimeRange	❌ Missing

Planned search Enhancements:

Feature	Purpose	Priority
`time_range` param	Filter by timestamp range	🟡 Medium
Search pagination / cursor	When results exceed `max_results` (capped at 1000), there's no cursor or offset to fetch the next page. Currently the only workaround is adding more filters to narrow results. A cursor/offset mechanism would allow iterating through large result sets.	🟡 Medium

Completed list_sources Enhancements:

Feature	Purpose	Version
Include total line count per source	Callers shouldn't need a separate call to know source size. Useful for calculating offsets, gauging search scope, etc.	✅ v0.6.0 (via get_stats)

Planned get_tail Enhancements:

Feature	Purpose	Priority
`since_line` parameter	Return only lines after a given line number. Enables efficient incremental polling of active sources without re-fetching or deduplicating.	✅ Complete

Planned get_lines Enhancements:

Feature	Purpose	Priority
Negative indexing / "from end" shorthand	Reading last N lines without knowing total_lines first (`get_tail` covers most cases, but minor friction when you need a specific offset from end)	🟢 Low

Completed Tools:

Tool	Purpose	Version
`get_stats`	Index metadata, severity breakdown, total lines, file size. Lightweight — reads index metadata only, no content scanning. Helps decide whether to tail or search, and whether a source is healthy.	✅ v0.6.0

Planned New Tools:

Tool	Purpose	Priority
`aggregate`	✅ Implemented via `search` query `aggregate` param. Count by field, top N. Both text queries (`count by (field) \| top N`) and MCP JSON work.	✅ Complete
`search_sources`	Search multiple sources at once, grouped results by source name. Essential for cross-service correlation (e.g., "find this request ID across all services"). Doesn't require timestamps or merging — just run the same query across all sources.	🔴 High
`fields`	Sample N lines from a source and return discovered field names, types, and example values. Makes structured queries far more usable — currently consumers must `get_tail` a few lines and visually parse JSON to discover field names before constructing a query. Critical for LLM consumers that can't eyeball the data.	🔴 High
`summarize`	Log overview: time range, top patterns, top services, error rate. Content-analysis based summary.	🟡 Medium
`add_source`	Register an existing log file as a named source. Lets AI agents dynamically add sources without editing config or piping data. CLI equivalent: `lazytail add <name> --path <path>`.	🟡 Medium
`export`	Dump filtered results to a file or return in bulk. Supports query filters, time range, and output format. Useful for "save me all errors from the last hour" workflows. TUI has export in backlog but MCP needs its own path since results are capped at 1000.	🟢 Low

Internal Improvements Done:

✅ Streaming filter with mmap (grep-like performance)
✅ SIMD-accelerated search (memchr/memmem)
✅ lines_searched tracking in FilterProgress::Complete
✅ Single-pass content extraction for matched lines
✅ Plain text output format (eliminates JSON escaping explosion for AI consumption)

Upcoming Focus Areas

Time Filtering & Aggregation 🔴 HIGH PRIORITY

Already Completed (landed post-v0.4.0):

✅ FilterQuery AST with serde derives (JSON interface for MCP)
✅ query parameter wired into MCP search tool
✅ Exclusion patterns (exclude field in query)
✅ logfmt parser support
✅ Nested field access (user.id)
✅ Text query parser for UI (json | level == "error")
✅ All comparison operators (eq, ne, regex, not_regex, contains, gt, lt, gte, lte)

Remaining:

Time range filtering (timestamp field detection, after/before)
~~Aggregation (count by (field), top N)~~ ✅ Complete
Additional aggregation types: avg, sum, min/max, percentiles, rate, count_distinct, histogram
Filter presets from config available in MCP

Sidecar Index & Combined Sources

Sidecar Index (.log.idx):

Binary index file alongside each captured log
Store arrival timestamp + byte offset per line
Append to index in real-time during capture
Header with validation: file size, mtime, first-4KB hash
Auto-rebuild on corruption/truncation detection
Enables time-based operations and merging

Combined Source View:

Merge multiple sources into single chronological view
Use sidecar timestamps for captured sources
Parse timestamps from log content for external files
Fallback to arrival order for streaming, concatenation for static
Source-colored lines or [SOURCE] prefix
Filter by source: source:API

MCP Project Scoping 🟡 MEDIUM PRIORITY

Goal: MCP server automatically scopes to the project when lazytail.yaml is present

Design Questions:

MCP server should detect lazytail.yaml by walking parent directories from CWD (same as TUI)
When project-scoped: list_sources returns project sources + config-defined sources
When global: list_sources returns ~/.config/lazytail/data/ sources (current behavior)
Sources from lazytail.yaml sources: definitions should appear alongside captured sources
Filter presets defined in config should be available via a list_presets or similar mechanism
Consider: should MCP expose both project and global sources, or only project when scoped?

Development Workflow

Before Starting a Feature

Update this roadmap with detailed tasks
Consider impact on existing tests
Plan for backward compatibility
Review CLAUDE.md for implementation guidance

During Development

Write tests first (TDD when appropriate)
Run pre-commit checks frequently
Keep commits focused and atomic
Update documentation as you go

Before Completion

All tests pass (cargo test)
Clippy clean (cargo clippy -- -D warnings)
Formatted (cargo fmt)
Documentation updated
Roadmap updated to mark task complete

Notes

This roadmap is a living document - update as priorities change
Focus on one major feature at a time
Keep production stability as top priority
User feedback will shape future direction

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

LazyTail Roadmap

Current Status (v0.6.0)

Upcoming Features & Improvements

🔴 HIGH PRIORITY

Phase 1: Multi-Tab Support (CLI Arguments) ✅

Phase 2: Source Discovery ✅

Phase 3: Source Capture Mode (Tee-like) ✅

Future Enhancements (Post-Phase 3)

Phase 5: Query Language (LogQL-style) 🔴 HIGHEST PRIORITY

Phase 4: Advanced Filter Modes

Explicit Query Filter Mode

🟡 MEDIUM PRIORITY

Expandable Log Entries ✅

Stats Panel (Left Column)

Log Format Detection & Severity Parsing ✅

Persist Filter History to Disk ✅

🟢 LOW PRIORITY

Mouse Controls

Multi-Line Selection

Search Highlighting

Scrollable Help Overlay ✅

Structured Logging & Debug Instrumentation 🔴 HIGH PRIORITY

Future Ideas (Backlog)

Performance & Scalability

Features

Project-Scoped Instances (lazytail.yaml) ✅

Developer Experience

MCP Tools Roadmap

Upcoming Focus Areas

Time Filtering & Aggregation 🔴 HIGH PRIORITY

Sidecar Index & Combined Sources

MCP Project Scoping 🟡 MEDIUM PRIORITY

Development Workflow

Before Starting a Feature

During Development

Before Completion

Notes

FilesExpand file tree

ARCHIVE.md

Latest commit

History

ARCHIVE.md

File metadata and controls

LazyTail Roadmap

Current Status (v0.6.0)

Upcoming Features & Improvements

🔴 HIGH PRIORITY

Phase 1: Multi-Tab Support (CLI Arguments) ✅

Phase 2: Source Discovery ✅

Phase 3: Source Capture Mode (Tee-like) ✅

Future Enhancements (Post-Phase 3)

Phase 5: Query Language (LogQL-style) 🔴 HIGHEST PRIORITY

Phase 4: Advanced Filter Modes

Explicit Query Filter Mode

🟡 MEDIUM PRIORITY

Expandable Log Entries ✅

Stats Panel (Left Column)

Log Format Detection & Severity Parsing ✅

Persist Filter History to Disk ✅

🟢 LOW PRIORITY

Mouse Controls

Multi-Line Selection

Search Highlighting

Scrollable Help Overlay ✅

Structured Logging & Debug Instrumentation 🔴 HIGH PRIORITY

Future Ideas (Backlog)

Performance & Scalability

Features

Project-Scoped Instances (lazytail.yaml) ✅

Developer Experience

MCP Tools Roadmap

Upcoming Focus Areas

Time Filtering & Aggregation 🔴 HIGH PRIORITY

Sidecar Index & Combined Sources

MCP Project Scoping 🟡 MEDIUM PRIORITY

Development Workflow

Before Starting a Feature

During Development

Before Completion

Notes