Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
455 lines (353 loc) · 15.8 KB

CONTRIBUTING.md

File metadata and controls

455 lines (353 loc) · 15.8 KB

Contributing to LazyTail

Thank you for your interest in contributing to LazyTail! This document provides guidelines and information for contributors.

Development Setup

Prerequisites

  • Rust (latest stable version)
  • Git

Building from Source

# Clone the repository
git clone https://github.com/raaymax/lazytail.git
cd lazytail

# Build in debug mode
cargo build

# Build in release mode
cargo build --release

# Run tests
cargo test

# Run with a log file
cargo run -- test.log

# Run in release mode with a log file
cargo run --release -- test.log

Project Architecture

src/
├── main.rs                   # Entry point, CLI (clap), and main event loop
├── lib.rs                    # Library crate interface (config, index, source, theme)
├── ansi.rs                   # ANSI escape sequence stripping
├── capture.rs                # Capture mode (tee-like stdin to file)
├── history.rs                # Filter history persistence
├── log_source.rs             # Domain state shared across TUI/Web/MCP (LogSource)
├── session.rs                # Session persistence (last-opened source per project)
├── signal.rs                 # Signal handling (SIGINT/SIGTERM)
├── source.rs                 # Source discovery and status tracking
├── app/
│   ├── mod.rs                # App struct, InputMode, ViewMode
│   ├── event.rs              # AppEvent enum
│   ├── tab.rs                # Per-tab TUI state (TabState)
│   └── viewport.rs           # Vim-style viewport scrolling
├── cache/
│   ├── mod.rs                # Cache module
│   ├── line_cache.rs         # LRU cache for line content
│   └── ansi_cache.rs         # ANSI-parsed line cache
├── cli/
│   ├── mod.rs                # CLI subcommand definitions
│   ├── init.rs               # `lazytail init` command
│   ├── bench.rs              # `lazytail bench` command
│   ├── config.rs             # `lazytail config` commands
│   ├── theme.rs              # `lazytail theme` commands
│   └── update.rs             # `lazytail update` command (feature-gated: self-update)
├── config/
│   ├── mod.rs                # Config module
│   ├── discovery.rs          # Walk parent dirs for lazytail.yaml
│   ├── loader.rs             # YAML config loading
│   ├── types.rs              # Config structs
│   └── error.rs              # Config error types
├── filter/
│   ├── mod.rs                # Filter trait
│   ├── aggregation.rs        # Grouped query results (count by)
│   ├── cancel.rs             # Cancellation tokens
│   ├── engine.rs             # Background filtering engine
│   ├── orchestrator.rs       # FilterOrchestrator — unified filter dispatch
│   ├── parallel_engine.rs    # Parallel filtering
│   ├── query.rs              # Query language (json | field == value)
│   ├── regex_filter.rs       # Regex filter implementation
│   ├── search_engine.rs      # SearchEngine — stateless search dispatch
│   ├── streaming_filter.rs   # Streaming mmap-based filter (grep-like)
│   └── string_filter.rs      # String matching filter
├── handlers/
│   ├── mod.rs                # Handler module
│   ├── input.rs              # Keyboard input handling
│   ├── filter.rs             # Filter progress handling
│   └── file_events.rs        # File change event handling
├── index/
│   ├── mod.rs                # Columnar index module
│   ├── builder.rs            # Index writer (flags, offsets, checkpoints)
│   ├── checkpoint.rs         # Checkpoint and SeverityCounts types
│   ├── column.rs             # ColumnReader/ColumnWriter for typed columnar storage
│   ├── flags.rs              # Severity enum, format flags, line classification
│   ├── lock.rs               # Advisory flock-based write lock
│   ├── meta.rs               # Index metadata (entry count, column bits)
│   └── reader.rs             # IndexReader — read-only access to flags/checkpoints
├── mcp/
│   ├── mod.rs                # MCP server module
│   ├── tools.rs              # Tool implementations (6 tools)
│   ├── types.rs              # Request/response types
│   ├── format.rs             # Plain text output formatters
│   └── ansi.rs               # ANSI escape code stripping
├── reader/
│   ├── mod.rs                # LogReader trait
│   ├── combined_reader.rs    # Multi-source chronological merging
│   ├── file_reader.rs        # Lazy file reader with line indexing
│   ├── huge_file_reader.rs   # Large file support (experimental)
│   ├── mmap_reader.rs        # Memory-mapped reader (experimental)
│   ├── sparse_index.rs       # Sparse line index
│   ├── stream_reader.rs      # Stdin/stream reader
│   └── tail_buffer.rs        # Tail buffer (experimental)
├── renderer/
│   ├── mod.rs                # PresetRegistry — compiled rendering presets
│   ├── builtin.rs            # Built-in preset definitions
│   ├── detect.rs             # Auto-detection of log format
│   ├── field.rs              # Field extraction from log lines
│   ├── format.rs             # Segment formatting
│   ├── preset.rs             # Compiled preset types
│   └── segment.rs            # Styled segment types
├── theme/
│   ├── mod.rs                # Theme struct, color parsing
│   └── loader.rs             # YAML theme loading, multi-format import
├── tui/
│   ├── mod.rs                # ratatui rendering coordinator
│   ├── aggregation_view.rs   # Aggregation result rendering
│   ├── help.rs               # Help overlay (keyboard shortcuts)
│   ├── log_view.rs           # Main log content rendering
│   ├── side_panel.rs         # Source tree panel
│   └── status_bar.rs         # Status bar rendering
├── update/                   # Self-update module (feature-gated: self-update)
│   ├── mod.rs                # Types, cache I/O, version comparison
│   ├── checker.rs            # GitHub release checking with cache
│   ├── installer.rs          # Binary download and replacement
│   └── detection.rs          # Package manager detection
├── watcher/
│   ├── mod.rs                # Watcher module exports
│   ├── file.rs               # File change detection via notify/inotify
│   └── dir.rs                # Directory watcher for source discovery
└── web/
    ├── mod.rs                # HTTP server (tiny_http)
    └── index.html            # Embedded SPA

For detailed architecture documentation, see CLAUDE.md.

Performance Characteristics

The viewer is designed to handle large log files efficiently:

  • Line indexing: O(n) one-time indexing, then O(1) random access
  • Viewport rendering: Only renders visible lines
  • Background filtering: Non-blocking filter execution in separate thread
  • Memory usage: ~constant regardless of file size (only viewport buffer in RAM)
  • Incremental filtering: Only filters new log lines when file grows

Dependencies

  • ratatui - TUI framework
  • crossterm - Cross-platform terminal manipulation
  • notify - File system watching
  • regex - Regular expression support
  • serde / serde_json - Serialization and JSON parsing
  • serde-saphyr - YAML config parsing
  • clap - CLI argument parsing
  • anyhow / thiserror - Error handling
  • ansi-to-tui - ANSI escape code parsing and color rendering
  • memmap2 - Memory-mapped file access
  • memchr - SIMD-accelerated byte searching
  • lru - LRU cache for line content
  • rayon - Parallel processing
  • signal-hook - Unix signal handling
  • dirs - Platform-specific directories
  • tiny_http - Lightweight HTTP server for web mode
  • colored - CLI colored output
  • strsim - Fuzzy matching for typo suggestions
  • xxhash-rust - Content hashing
  • unicode-width - Text width calculation
  • libc - Unix-specific operations (flock, etc.)
  • rmcp - MCP server framework (optional, feature-gated)
  • schemars - JSON Schema generation for MCP (optional)
  • tokio - Async runtime for MCP server (optional)
  • self_update - GitHub release checking and binary replacement (optional, feature-gated)

Development Tools

# Run clippy for linting
cargo clippy

# Check code formatting
cargo fmt -- --check

# Format code
cargo fmt

Testing

Running Tests

# Run all fast tests (default - takes ~0.1s)
cargo test

# Run all tests including slow/integration tests
cargo test -- --include-ignored

# Run only slow tests
cargo test -- --ignored

# Run tests for a specific module
cargo test filter
cargo test watcher

Fast vs Slow Tests:

  • Fast tests (default): Unit tests that run in milliseconds
  • Slow tests (marked with #[ignore]): Integration tests involving real file system operations, timing-sensitive tests, or stress tests
  • CI runs fast tests by default to keep builds quick
  • Developers can optionally run slow tests locally for thorough validation

Test Log Files

The repository includes test log files and generators:

  • test.log - Plain text logs with various log levels (INFO, DEBUG, WARN, ERROR)
  • generate_logs.sh - Script to generate plain text logs continuously
  • generate_colored_logs.sh - Script to generate ANSI-colored logs continuously

Testing Live Reload and Follow Mode

Test the file watching feature:

# Terminal 1: Start generating logs
./generate_logs.sh live_test.log

# Terminal 2: Watch the logs in real-time
cargo run --release -- live_test.log

Press f to enable follow mode and watch new lines scroll into view automatically.

Testing with Colored Logs

Test ANSI color support:

# Terminal 1: Generate colored logs
./generate_colored_logs.sh live_test_colored.log

# Terminal 2: View the logs with full color rendering
cargo run --release -- live_test_colored.log

Commit Convention

This project uses Conventional Commits for automatic changelog generation and version management.

Commit Message Format

<type>: <description>

[optional body]

[optional footer(s)]

Commit Types

  • feat: A new feature (bumps minor version: 0.1.0 → 0.2.0)
  • fix: A bug fix (bumps patch version: 0.1.0 → 0.1.1)
  • docs: Documentation only changes
  • style: Changes that don't affect code meaning (whitespace, formatting)
  • refactor: Code change that neither fixes a bug nor adds a feature
  • perf: Performance improvement
  • test: Adding or modifying tests
  • chore: Changes to build process or auxiliary tools

Examples

# Feature addition
git commit -m "feat: add JSON log parsing support"

# Bug fix
git commit -m "fix: resolve viewport scrolling issue in filtered mode"

# Documentation
git commit -m "docs: update installation instructions"

# Breaking change (bumps major version: 0.1.0 → 1.0.0)
git commit -m "feat: redesign filter API

BREAKING CHANGE: Filter trait now requires async implementation"

CI/CD Workflows

The project uses GitHub Actions for continuous integration and automated releases.

CI Workflow (.github/workflows/ci.yml)

Runs on every push and pull request to the master branch.

Jobs:

  • Test: Runs on Linux and macOS
    • Executes all tests with cargo test --verbose
    • Runs cargo clippy for linting
    • Checks code formatting with cargo fmt
  • Build: Creates artifacts for all platforms
    • Linux x86_64
    • macOS x86_64 (Intel)
    • macOS aarch64 (Apple Silicon)

The CI must pass before PRs can be merged.

Release PR Workflow (.github/workflows/release-pr.yml)

Runs automatically on every push to the master branch.

Functionality:

  • Uses release-please to automate releases
  • Analyzes commit messages since the last release
  • Creates or updates a release PR with:
    • Updated CHANGELOG.md with all changes
    • Version bump in Cargo.toml based on commit types
    • Generated release notes

Version Bumping:

  • feat: commits → minor version bump (0.1.0 → 0.2.0)
  • fix: commits → patch version bump (0.1.0 → 0.1.1)
  • BREAKING CHANGE: → major version bump (0.1.0 → 1.0.0)

Release Workflow (.github/workflows/release.yml)

Triggered when the release PR is merged (release is published).

Jobs:

  • Builds optimized, stripped binaries for all platforms
  • Compresses binaries as .tar.gz archives
  • Uploads artifacts to the GitHub release:
    • lazytail-linux-x86_64.tar.gz
    • lazytail-macos-x86_64.tar.gz
    • lazytail-macos-aarch64.tar.gz

Release Process

The release process is fully automated:

1. Make Commits

Commit your changes to the master branch using conventional commit messages:

git commit -m "feat: add new filtering mode"
git commit -m "fix: correct ANSI color parsing"
git push origin master

2. Release PR Created

After your commits are pushed:

  • The release-please workflow automatically runs
  • A release PR is created or updated (if one already exists)
  • The PR includes:
    • All changes since the last release in CHANGELOG.md
    • Version bump in Cargo.toml
    • Generated release notes

3. Review and Merge

When ready to publish a release:

  1. Review the auto-generated release PR
  2. Verify the changelog and version bump are correct
  3. Merge the release PR

4. Release Published

After merging the release PR:

  • A new GitHub release is automatically created with a version tag
  • The release workflow builds binaries for all platforms
  • Binaries are uploaded to the release page
  • Users can download the new release

Pre-Commit Checklist

All of the following MUST pass before committing any changes:

# 1. Format code
cargo fmt

# 2. Run linter and fix any warnings
cargo clippy

# 3. Verify types compile
cargo check

# 4. Run tests
cargo test

These checks are enforced by CI - PRs will not be merged if any check fails.

Pull Request Guidelines

  1. Create a feature branch: Don't commit directly to master

    git checkout -b feature/your-feature-name

  2. Write conventional commits: Follow the commit convention outlined above

  3. Add tests: If adding new functionality, include tests

  4. Run pre-commit checks: Before pushing, ensure all checks pass (see Pre-Commit Checklist above)

  5. Write clear PR descriptions: Explain what the PR does and why

  6. Keep PRs focused: One feature/fix per PR when possible

Code Style

  • Follow Rust standard formatting (cargo fmt)
  • Address all clippy warnings (cargo clippy)
  • Write clear, self-documenting code
  • Add comments only when the logic isn't self-evident
  • Keep functions focused and reasonably sized

Testing

  • Write unit tests for new functions and modules
  • Test edge cases and error conditions
  • Run the full test suite before submitting PRs
  • Manually test with various log files when changing UI or filtering logic

Architecture Guidelines

See CLAUDE.md for detailed architecture documentation including:

  • Core event loop design
  • State management patterns
  • Viewport scrolling system
  • Reader architecture
  • Filter system
  • File watching implementation

Questions or Issues?

  • Open an issue on GitHub for bugs or feature requests
  • Check existing issues before creating new ones
  • Provide detailed reproduction steps for bugs
  • Include log samples when reporting filtering or parsing issues

License

By contributing to LazyTail, you agree that your contributions will be licensed under the same license as the project.