feat(usage): normalize and report token usage metrics

- Add `UsageExtractor` to normalize raw usage data according to vendor mappings in `resources/models.yaml`
- Update `ModelDriverInterface` signature and all drivers (`Anthropic`, `Google`, `OpenAi`, `Yandex`, `DeepL`) to return both `text` and `usage`
- Adapt `Bblslug::translate()` to extract and include usage metrics in its result
- Enhance `runFromCli()` to display a structured “Usage metrics” section (total + breakdown) after translation
- Update and slightly restructure `README.md` text and CLI sections to document the new usage-metrics feature

Author: bionicman

README.md

@@ -7,34 +7,35 @@ It leverages LLM-based APIs to translate plain text or HTML while preserving str
 APIs supported:
 
 - Anthropic (Claude):
- - `anthropic:claude-haiku-3.5` - Claude Haiku 3.5 (latest)
- - `anthropic:claude-opus-4` - Claude Opus 4 (20250514)
- - `anthropic:claude-sonnet-4` - Claude Sonnet 4 (20250514)
-- DeepL
- - `deepl:free` - DeepL free tier
- - `deepl:pro` - DeepL pro tier
-- Google (Gemini)
- - `google:gemini-2.0-flash` - Gemini 2.0 Flash
- - `google:gemini-2.5-flash` - Gemini 2.5 Flash
- - `google:gemini-2.5-flash-lite` - Gemini 2.5 Flash Lite
- - `google:gemini-2.5-pro` - Gemini 2.5 Pro
-- OpenAI (GPT)
- - `openai:gpt-4` - OpenAI GPT-4
- - `openai:gpt-4-turbo` - OpenAI GPT-4 Turbo
- - `openai:gpt-4o` - OpenAI GPT-4o
- - `openai:gpt-4o-mini` - OpenAI GPT-4o Mini
+  - `anthropic:claude-haiku-3.5` - Claude Haiku 3.5 (latest)
+  - `anthropic:claude-opus-4` - Claude Opus 4 (20250514)
+  - `anthropic:claude-sonnet-4` - Claude Sonnet 4 (20250514)
+- DeepL:
+  - `deepl:free` - DeepL free tier
+  - `deepl:pro` - DeepL pro tier
+- Google (Gemini):
+  - `google:gemini-2.0-flash` - Gemini 2.0 Flash
+  - `google:gemini-2.5-flash` - Gemini 2.5 Flash
+  - `google:gemini-2.5-flash-lite` - Gemini 2.5 Flash Lite
+  - `google:gemini-2.5-pro` - Gemini 2.5 Pro
+- OpenAI (GPT):
+  - `openai:gpt-4` - OpenAI GPT-4
+  - `openai:gpt-4-turbo` - OpenAI GPT-4 Turbo
+  - `openai:gpt-4o` - OpenAI GPT-4o
+  - `openai:gpt-4o-mini` - OpenAI GPT-4o Mini
 - Yandex:
   - `yandex:gpt-lite` - YandexGPT Lite
   - `yandex:gpt-pro` - YandexGPT Pro
   - `yandex:gpt-32k` - YandexGPT Pro 32K
 
 ## Features
 
-- Supports **html** and **plain text** (`--format=text|html`)
+- Supports **HTML** and **plain text** (`--format=text|html`)
 - Placeholder-based protection with filters: `html_pre`, `html_code`, `url`, etc.
-- Model selection via `--model=vendor:name` (`deepl:pro`, `google:gemini-2.5-flash`, `openai:gpt-4o`, …)
-- Fully configurable backend registry
+- Model selection via `--model=vendor:name`
+- Fully configurable backend registry (via `resources/models.yaml`)
 - **Dry-run** mode to preview placeholders without making API calls
+- **Variables** (`--variables`) to send or override model-specific options
 - **Verbose** mode (`--verbose`) to print request previews
 - Can be invoked as a CLI tool or embedded in PHP code
 
@@ -47,42 +48,45 @@ chmod +x vendor/bin/bblslug
 
 ## CLI Usage
 
+### Prepare
+
 1. **Always specify a model** with `--model=vendor:name` option.
 
 2. **Export your API key(s)** before running:
 
- ```bash
- export ANTHROPIC_API_KEY=...
- export DEEPL_FREE_API_KEY=...
- export DEEPL_PRO_API_KEY=...
- export GOOGLE_API_KEY=...
- export OPENAI_API_KEY=...
- export YANDEX_API_KEY=... && export YANDEX_FOLDER_ID=...
- ```
+  ```bash
+  export ANTHROPIC_API_KEY=...
+  export DEEPL_FREE_API_KEY=...
+  export DEEPL_PRO_API_KEY=...
+  export GOOGLE_API_KEY=...
+  export OPENAI_API_KEY=...
+  export YANDEX_API_KEY=... && export YANDEX_FOLDER_ID=...
+  ```
 
-**NB!** Some vendors require additional parameters for client authentication (like Yandex)
+  **NB!** Some vendors require additional parameters, e.g. `YANDEX_FOLDER_ID`.
 
 3. **Input / output**:
 
- - If `--source` is omitted, Bblslug reads from **STDIN**.
- - If `--translated` is omitted, Bblslug writes to **STDOUT**.
+  - If `--source` is omitted, Bblslug reads from **STDIN**.
+  - If `--translated` is omitted, Bblslug writes to **STDOUT**.
 
 4. **Optional proxy**:
 
-To route requests through a proxy (e.g. HTTP or SOCKS5), use the `--proxy` option or set the `BBLSLUG_PROXY` environment variable:
-
-```bash
-# using CLI flag
-vendor/bin/bblslug --proxy="http://localhost:8888" ...
+  To route requests through a proxy (e.g. HTTP or SOCKS5), use the `--proxy` option or set the `BBLSLUG_PROXY` environment variable:
 
-# or set it globally
-export BBLSLUG_PROXY="socks5h://127.0.0.1:9050"
-```
+  ```bash
+  # using CLI flag
+  vendor/bin/bblslug --proxy="http://localhost:8888" ...
+  
+  # or set it globally
+  export BBLSLUG_PROXY="socks5h://127.0.0.1:9050"
+  ```
 
-This works for all HTTP requests and supports authentication (`http://user:pass@host:port`).
+  This works for all HTTP requests and supports authentication (`http://user:pass@host:port`).
 
 
 ### Show available models
+
 ```bash
 vendor/bin/bblslug --list-models
 ```
@@ -121,32 +125,17 @@ vendor/bin/bblslug \
   --context="Translate as a professional technical translator"
 ```
 
-### Pipe STDIN → file
+### Pass model-specific variables
 
 ```bash
 vendor/bin/bblslug \
   --model=vendor:name \
   --format=text \
-  --source=input.txt
-```
-
-### Pipe STDIN → file
-
-```bash
-cat input.txt | vendor/bin/bblslug \
-  --model=vendor:name \
-  --format=text \
+  --variables=foo=bar,foo2=bar2 \
+  --source=in.txt \
   --translated=out.txt
 ```
 
-### Pipe STDIN → STDOUT
-
-```bash
-echo "Hello world" | vendor/bin/bblslug \
-  --model=vendor:name \
-  --format=text
-```
-
 ### Dry-run placeholders only
 
 ```bash
@@ -169,69 +158,127 @@ vendor/bin/bblslug \
   --translated=out.html
 ```
 
-### Use additional model/vendor-specific options
+### Pipe STDIN → file
 
 ```bash
-vendor/bin/bblslug \
+cat input.txt | vendor/bin/bblslug \
   --model=vendor:name \
-  --variables=some=XXX,other=YYY \
-  --format=html \
-  --verbose \
-  --source=input.html \
-  --translated=out.html
+  --format=text \
+  --translated=out.txt
+```
+
+### Pipe STDIN → STDOUT
+
+```bash
+echo "Hello world" | vendor/bin/bblslug \
+  --model=vendor:name \
+  --format=text > translated.out
 ```
 
+### Statistics
+
+- **Usage metrics**
+
+  After each translation (when not in dry-run), Bblslug prints to stderr a summary of consumed usage metrics, for example:
+
+  ```
+  Usage metrics:
+      Tokens:
+          Total:       1074
+          -----------------
+          Prompt:      631
+          Completion:  443
+  ```
+
 ## PHP Library Usage
 
-You can embed Bblslug in your PHP project:
+You can embed Bblslug in your PHP project.
+
+### Quickstart
+
+1. **Install:**
+
+  ```bash
+  composer require habr/bblslug
+  ```
+
+2. **Require & Import:**
+
+  ```php
+  require 'vendor/autoload.php';
+  use Bblslug\Bblslug;
+  ```
+
+3. **Translate:**
+
+  ```php
+  $text = file_get_contents('input.html');
+  $result = Bblslug::translate(
+      apiKey:   getenv('MODEL_API_KEY'),         // API key for the chosen model
+      format:   'html',                          // 'text' or 'html'
+      modelKey: 'vendor:model',                  // Model identifier (e.g. deepl:free, openai:gpt-4o, etc.)
+      text:     $text,                           // Source text or HTML
+      // optional:
+      // Additional context/prompt pass to model
+      context:    'Translate as a professional technical translator',
+      filters:    ['url','html_code'],           // List of placeholder filters
+      proxy:      getenv('BBLSLUG_PROXY'),       // Optional proxy URI (http://..., socks5h://...)
+      sourceLang: 'DE',                          // Source language code (optional; autodetect if null)
+      targetLang: 'EN',                          // Target language code (optional; default from driver settings)
+      variables:  ['foo'=>'bar'],                // model-specific overrides
+      verbose:    true,                          // If true, returns debug request/response
+  );
+  echo $result['result'];
+  ```
+
+### Result structure
 
 ```php
-<?php
-require 'vendor/autoload.php';
-
-use Bblslug\Bblslug;
-
-// Load input text or HTML from file
-$text   = file_get_contents('input.html');
-
-// Call library translate method
-$result = Bblslug::translate(
-    apiKey:     getenv('DEEPL_PRO_API_KEY'),     // API key for the chosen model
-    format:     'html',                          // 'text' or 'html'
-    modelKey:   'deepl:pro',                     // Model identifier (e.g. deepl:free, deepl:pro, openai:gpt-4o)
-    text:       $text,                           // Source text or HTML
-
-    // optional parameters:
-    context:    null,                            // Additional context/prompt (DeepL: context)
-    dryRun:     false,                           // If true, only prepare placeholders, no API call
-    filters:    ['url', 'html_code'],            // List of placeholder filters
-    proxy:      getenv('BBLSLUG_PROXY'),         // Optional proxy URI (http://..., socks5h://...)
-    sourceLang: null,                            // Source language code (optional; autodetect if null)
-    targetLang: null,                            // Target language code (optional; default from driver settings)
-    variables:  TBD,                             // TBD
-    verbose:    true,                            // If true, prints debug request/response to stderr
-);
-
-// Result output example
-// $result = [
-//   'original'    => '...',      // Original input
-//   'prepared'    => '...',      // After placeholder filters
-//   'result'      => '...',      // Translated result
-//   'lengths'     => [           // Character counts
-//     'original'   => 123,
-//     'prepared'   => 100,
-//     'translated' => 130
-//   ],
-//   'filterStats' => [           // Placeholder stats
-//     ['filter'=>'url','count'=>5], …
-// ];
-
-echo $result['result'];
+[
+  'original'        => string,   // Original input
+  'prepared'        => string,   // After placeholder filters
+  'result'          => string,   // Translated result
+  'httpStatus'      => int,      // HTTP status
+  'debugRequest'    => string,   // Request debug
+  'debugResponse'   => string,   // Response debug
+  'rawResponseBody' => string,   // Response body
+  'consumed'        => [                  // Normalized usage metrics
+     'tokens' => [
+       'total'     => int,                // Total tokens consumed
+       'breakdown' => [                   // Per-type breakdown
+         'prompt'      => int,            // Name depeds of model
+         'completion'  => int,            // Name depeds of model
+       ],
+     ],
+     // additional categories if supported by the model...
+  'lengths'         => [         // Text length statistics
+     'original'    => int,       // - original text
+     'prepared'    => int,       // - after placeholder filters
+     'translated'  => int,       // - returned translated text
+  ],
+  'filterStats'     => [         // Placeholder stats
+     ['filter'=>'url','count'=>3], …
+  ],
+]
 ```
 
-## Examples
+### Error handling
+
+```php
+try {
+    $res = Bblslug::translate(...);
+} catch (\InvalidArgumentException $e) {
+    // invalid model, missing API key, etc.
+} catch (\RuntimeException $e) {
+    // HTTP error, parse failure, driver-specific error
+}
+```
+
+
+
+## Samples
 
-You can find sample input files under the `examples/` directory.
+You can find sample input files under the `samples/` directory.
 
 ## License
 

resources/models.yaml

@@ -21,6 +21,12 @@ anthropic:
   limits:
     estimated_max_chars: 400000
   http_error_handling: true
+  usage:
+    tokens:
+      total: total_tokens
+      breakdown:
+        prompt: prompt_tokens
+        completion: completion_tokens
 
   models:
     claude-haiku-3.5:
@@ -107,6 +113,12 @@ google:
       - system_instruction
       - contents
       - generationConfig
+  usage:
+    tokens:
+      total: totalTokenCount
+      breakdown:
+        prompt: promptTokenCount
+        candidates: candidatesTokenCount
 
   models:
     gemini-2.0-flash:
@@ -180,6 +192,12 @@ openai:
     max_tokens: 128000
     token_estimator: gpt
     estimated_max_chars: 512000
+  usage:
+    tokens:
+      total: total_tokens
+      breakdown:
+        prompt: prompt_tokens
+        completion: completion_tokens
 
   models:
     gpt-4:
@@ -232,6 +250,12 @@ yandex:
     token_estimator: gpt
     estimated_max_chars: 32768
   http_error_handling: true
+  usage:
+    tokens:
+      total: totalTokens
+      breakdown:
+        input: inputTextTokens
+        completion: completionTokens
 
   models:
     gpt-lite: