v0.0.3

Release v0.0.3

Installation

Homebrew (macOS and Linux)

brew install agentstation/tap/tokenizer

Download Binary

Download the appropriate binary for your platform from the assets below.

Using Go

go install github.com/agentstation/tokenizer/cmd/tokenizer@v0.0.3

Verify Checksums

# Download checksums.txt and the binary, then:
sha256sum -c checksums.txt

Changelog

New Features

• 6f72418: feat: add automatic streaming detection for piped input (@jackspirou)
• 72a1350: feat: make encode the default action for llama3 command (@jackspirou)

Bug fixes

• 0fcfc76: fix: simplify release naming to just version tag (@jackspirou)