kafkagoat

______        _____________                             _____ 
 ___  /_______ ___  __/__  /_______ _______ ____________ __  /_
 __  //_/  __ `/_  /_ __  //_/  __ `/_  __ `/_  __ `/  __ \  __ `/  __/
 _  ,<  / /_/ /_  __/ _  ,<  / /_/ /_  /_/ // /_/ / /_/ // /_  
/_/|_| \__,_/ /_/    /_/|_| \__,_/ _\__, / \____/\__,_/ \__/  
                                   /____/                   

A terminal-based Kafka client for developers and operators. Inspect, debug, and monitor Kafka topics from your terminal.

This project was developed with the assistance of AI agents, in order to practice the inclusion of this technology in the development process.

Built with Bubble Tea for the TUI and franz-go for reliable Kafka connectivity. Using zalando/go-keyring for securely storing credentials.

What is kafkagoat?

kafkagoat is a lightweight, terminal-based Kafka consumer designed for developers working with event-driven systems and operators who need to monitor Kafka clusters. It provides a keyboard-driven interface for browsing messages across topics, inspecting payloads and headers, and filtering message streams in real-time.

Unlike web-based tools, kafkagoat runs entirely in your terminal, making it ideal for SSH connections to remote environments, CI/CD debugging sessions, or quick ad-hoc inspections without leaving your development workflow.

Features

• Multi-session tabs: Connect to multiple Kafka clusters at once, switch with Alt+1 through Alt+9
• Advanced message filtering: Query language supporting exact match, regex, comparisons, and time-based filters
• Safe decoding: Automatic gzip decompression and JSON pretty-printing with graceful fallbacks
• Authentication: TLS support and SASL mechanisms (plain, SCRAM-SHA-256, SCRAM-SHA-512)
• Visual themes: 4 built-in color schemes (Monokai Spectre, One Dark, One Light, Solarized Light)
• Session persistence: Remember connection configurations without storing passwords
• Pause/resume: Temporarily halt message consumption to inspect the current buffer
• Flexible offset starting: Begin consumption from latest, earliest, or a relative time offset

Installation

Pre-built binaries are available on the GitHub Releases page.

Install from Release

1. Download the archive for your platform:

   • Linux: kgoat-linux-amd64.tar.gz
   • macOS: kgoat-darwin-amd64.tar.gz or kgoat-darwin-arm64.tar.gz (Apple Silicon)
   • Windows: kgoat-windows-amd64.zip

2. Extract the archive:

   tar -xzf kgoat-linux-amd64.tar.gz
   # or for Windows
   unzip kgoat-windows-amd64.zip

3. Install the binary:

   # Move to a directory in your PATH
   mv kgoat /usr/local/bin/
   
   # Or add to PATH temporarily
   export PATH="$PATH:$(pwd)/kgoat"

Build from Source

git clone https://github.com/yourusername/kafkagoat
cd kafkagoat
go build -o kgoat .
sudo mv kgoat /usr/local/bin/

Quick Start

1. Launch kgoat: kgoat
2. Enter your Kafka brokers and topics
3. Optionally configure TLS, SASL, and consumer group settings
4. Press Enter to connect and start consuming messages
5. Use arrow keys to navigate, Enter to inspect a message

Usage

Keyboard Shortcuts

Key	Action
?	Toggle help
Ctrl+T	Cycle theme
Ctrl+N	Open session manager
Esc	Go back / close modals
Ctrl+Q	Quit

Message Table

Key	Action
Up/Down or k/j	Navigate messages
Enter or Space	Inspect selected message
/	Activate filter mode
p	Pause/resume consumption
Ctrl+L	Clear all messages
PgUp/PgDn or b/f	Page up/down
Home/End or g/G	Go to top/bottom

Tab Management

Key	Action
Alt+1 - Alt+9	Switch to specific tab
Alt+N	Open new session
Alt+W	Close current tab
Alt+Left/Right	Cycle through tabs

Detail View

Key	Action
Esc / Q / Space	Close detail view
C	Copy payload to clipboard
H	Copy headers to clipboard
Left/Right or h/l	Previous/next message

Filtering Messages

Enter filter mode with / and construct queries using:

field operator value AND ...

Fields: topic, pkey, payload, headers, partition, offset, size, time, encoding, format

Operators:

Operator	Meaning
=	Exact match
*	Contains
~	Regex match
>, >=, <, <=	Comparison
BETWEEN	Range query

Examples:

topic=orders AND payload*error
payload~"timeout|failed" AND time>-1d
partition=0 AND size>1024

Time values (UTC):

• -1d = yesterday same time
• h14 = today at 14:00
• h1430 = today at 14:30
• yh14 = yesterday at 14:00
• 241231h1200 = 2024-12-31 12:00
• RFC3339 format also accepted

Session Management

• Saved sessions appear on the welcome screen
• Enter connects to a saved session
• d duplicates a session as a template
• Delete removes a saved session
• Sessions are saved after successful connection if "Remember" is checked

Configuration

Saved sessions are stored at:

• Linux/macOS: $XDG_CONFIG_HOME/kafkagoat/ or ~/.config/kafkagoat/
• Windows: %APPDATA%\kafkagoat\

Passwords are never saved. If credentials are saved, they are stored in the system's credential manager.

Building from Source

If you prefer to build from source:

git clone https://github.com/yourusername/kafkagoat
cd kafkagoat
go build -o kgoat .

Local Kafka for Development

The repository includes a docker-compose.yml for a single-node Redpanda broker:

docker compose up -d
docker compose exec redpanda rpk topic create kafkagoat

See docs/redpanda.md for helper scripts to produce test messages.

Running Tests

go test ./...

Contributing

The feature set is considered mostly complete, but there's significant room for quality-of-life improvements and code refactoring, particularly in the TUI layer. Contributions welcome in areas such as:

• Enhanced filtering capabilities
• Improved keyboard navigation and discoverability
• Additional themes and customization options
• Code organization and test coverage
• Better error handling
• UI/UX refactorings and improvements
• Performance optimizations

Please feel free to submit pull requests or open issues for discussion.

License

BSD 3-Clause License. See the LICENSE file for details.