feat: add progress_format option for machine-readable JSON output

[podarok]

Summary

Add a progress_format option to BpeTrainer that allows choosing between different progress output formats:

• Indicatif (default): Interactive terminal progress bars (current behavior, unchanged)
• JsonLines: Machine-readable JSON lines to stderr
• Silent: No progress output

This enables programmatic consumption of training progress for web UIs, logging systems, and other non-TTY environments where indicatif progress bars are not visible.

Motivation

When running tokenizer training in a web application backend or logging environment, the indicatif progress bars:

1. Don't display (auto-hidden when stderr is not a TTY)
2. Can't be parsed programmatically even if captured

This PR adds an opt-in JSON output mode that emits structured progress data:

{"stage":"Tokenize words","current":1000,"total":5000000}
{"stage":"Count pairs","current":500,"total":5000000}
{"stage":"Compute merges","current":30000,"total":65536}

Changes

Rust Core

• Add ProgressFormat enum to tokenizers/src/utils/progress.rs
• Export ProgressFormat from tokenizers/src/utils/mod.rs and tokenizers/src/lib.rs
• Add progress_format field and .progress_format() builder method to BpeTrainer
• Modify setup_progress() to only create indicatif bar when format is Indicatif
• Add emit_json_progress() helper that outputs JSON when format is JsonLines
• Add get_word_count() method to BpeTrainer for progress estimation

Python Bindings

• Add progress_format parameter to BpeTrainer constructor (accepts "indicatif", "json", "silent")
• Add progress_format getter/setter properties
• Add get_word_count() method

Usage

from tokenizers.trainers import BpeTrainer

# Current usage - unchanged
trainer = BpeTrainer(vocab_size=65536, show_progress=True)

# New: machine-readable JSON output
trainer = BpeTrainer(vocab_size=65536, progress_format="json")

# Or set after creation
trainer.progress_format = "json"

Backward Compatibility

• Default behavior is Indicatif - identical to current behavior
• Existing code works without any changes
• New option is opt-in only

Test Plan

• Rust core compiles
• Python bindings build
• Default format shows indicatif progress bars
• JSON format outputs valid JSON lines to stderr
• Silent format produces no output
• get_word_count() returns correct count after feeding

[podarok]

works great with this change