Skip to content

Commit b30e49c

Browse files
hydropixhydropix
committed
docs: Unify OpenAI-compatible provider terminology and fix wiki links
- Rename "LM Studio" references to "OpenAI-Compatible" to support llama.cpp, vLLM, LocalAI, etc. - Update UI, API routes, and error messages to reflect broader local server compatibility - Fix benchmark wiki template links to use GitHub wiki format (no .md extension) - Add BENCHMARK.md documentation for the translation quality benchmark system - Update PROVIDERS.md with server endpoint table and improved setup instructions
1 parent b64eb28 commit b30e49c

File tree

17 files changed

+520
-82
lines changed

17 files changed

+520
-82
lines changed

‎README.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -45,12 +45,12 @@ The web interface opens at **http://localhost:5000**
4545
| Provider | Type | Setup |
4646
|----------|------|-------|
4747
| **Ollama** | Local | [ollama.com](https://ollama.com/) |
48-
| **LM Studio** | Local (OpenAI-compatible) | [lmstudio.ai](https://lmstudio.ai/) |
48+
| **OpenAI-Compatible** | Local | llama.cpp, LM Studio, vLLM, LocalAI... |
4949
| **OpenRouter** | Cloud (200+ models) | [openrouter.ai/keys](https://openrouter.ai/keys) |
5050
| **OpenAI** | Cloud | [platform.openai.com](https://platform.openai.com/api-keys) |
5151
| **Gemini** | Cloud | [Google AI Studio](https://makersuite.google.com/app/apikey) |
5252

53-
> **LM Studio / Local OpenAI-compatible servers:** Use `--provider openai` with `--api_endpoint http://localhost:1234/v1/chat/completions`
53+
> **OpenAI-Compatible servers:** Use `--provider openai` with your server's endpoint (e.g., llama.cpp: `http://localhost:8080/v1/chat/completions`, LM Studio: `http://localhost:1234/v1/chat/completions`)
5454
5555
See [docs/PROVIDERS.md](docs/PROVIDERS.md) for detailed setup instructions.
5656

@@ -88,9 +88,9 @@ python translate.py -i book.txt -o book_fr.txt --provider openai \
8888
python translate.py -i book.txt -o book_fr.txt --provider gemini \
8989
--gemini_api_key YOUR_KEY -m gemini-2.0-flash
9090

91-
# With LM Studio (or any OpenAI-compatible local server)
91+
# With local OpenAI-compatible server (llama.cpp, LM Studio, vLLM, etc.)
9292
python translate.py -i book.txt -o book_fr.txt --provider openai \
93-
--api_endpoint http://localhost:1234/v1/chat/completions -m your-model
93+
--api_endpoint http://localhost:8080/v1/chat/completions -m your-model
9494
```
9595

9696
### Main Options

‎benchmark/wiki/generator.py‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -270,15 +270,15 @@ def _generate_all_languages_page(self, run: BenchmarkRun) -> None:
270270
lang_info = self._get_language_info(stats.language_code)
271271
indicator = get_score_indicator(stats.avg_overall)
272272
rows.append([
273-
f"[{lang_info['name']}](languages/{self._slugify(lang_info['name'])}.md)",
273+
f"[{lang_info['name']}](languages/{self._slugify(lang_info['name'])})",
274274
lang_info['native_name'],
275275
lang_info['category'],
276276
f"{indicator} {stats.avg_overall:.1f}",
277277
stats.best_model or "N/A",
278278
])
279279

280280
table = format_markdown_table(headers, rows)
281-
content = f"# All Languages\n\n{table}\n\n---\n\n[< Back to Home](Home.md)\n"
281+
content = f"# All Languages\n\n{table}\n\n---\n\n[< Back to Home](Home)\n"
282282

283283
(self.output_dir / "All-Languages.md").write_text(content, encoding="utf-8")
284284

@@ -292,7 +292,7 @@ def _generate_all_models_page(self, run: BenchmarkRun) -> None:
292292
for stats in sorted(model_stats, key=lambda x: x.avg_overall, reverse=True):
293293
indicator = get_score_indicator(stats.avg_overall)
294294
rows.append([
295-
f"[{stats.model}](models/{self._slugify(stats.model)}.md)",
295+
f"[{stats.model}](models/{self._slugify(stats.model)})",
296296
f"{indicator} {stats.avg_overall:.1f}",
297297
f"{stats.avg_accuracy:.1f}",
298298
f"{stats.avg_fluency:.1f}",
@@ -301,7 +301,7 @@ def _generate_all_models_page(self, run: BenchmarkRun) -> None:
301301
])
302302

303303
table = format_markdown_table(headers, rows)
304-
content = f"# All Models\n\n{table}\n\n---\n\n[< Back to Home](Home.md)\n"
304+
content = f"# All Models\n\n{table}\n\n---\n\n[< Back to Home](Home)\n"
305305

306306
(self.output_dir / "All-Models.md").write_text(content, encoding="utf-8")
307307

‎benchmark/wiki/templates/home.md.j2‎

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ Overall performance across all tested languages:
2323
| Rank | Model | Avg Score | Accuracy | Fluency | Style | Languages Tested |
2424
|------|-------|-----------|----------|---------|-------|------------------|
2525
{% for model in model_rankings %}
26-
| {{ loop.index }} | [{{ model.name }}](models/{{ model.slug }}.md) | {{ model.indicator }} {{ model.avg_overall|round(1) }} | {{ model.avg_accuracy|round(1) }} | {{ model.avg_fluency|round(1) }} | {{ model.avg_style|round(1) }} | {{ model.languages_tested }} |
26+
| {{ loop.index }} | [{{ model.name }}](models/{{ model.slug }}) | {{ model.indicator }} {{ model.avg_overall|round(1) }} | {{ model.avg_accuracy|round(1) }} | {{ model.avg_fluency|round(1) }} | {{ model.avg_style|round(1) }} | {{ model.languages_tested }} |
2727
{% endfor %}
2828

2929
---
@@ -35,11 +35,11 @@ Best translation quality by target language:
3535
| Rank | Language | Native | Avg Score | Best Model | Tests |
3636
|------|----------|--------|-----------|------------|-------|
3737
{% for lang in language_rankings[:15] %}
38-
| {{ loop.index }} | [{{ lang.name }}](languages/{{ lang.slug }}.md) | {{ lang.native_name }} | {{ lang.indicator }} {{ lang.avg_overall|round(1) }} | {{ lang.best_model }} | {{ lang.total_translations }} |
38+
| {{ loop.index }} | [{{ lang.name }}](languages/{{ lang.slug }}) | {{ lang.native_name }} | {{ lang.indicator }} {{ lang.avg_overall|round(1) }} | {{ lang.best_model }} | {{ lang.total_translations }} |
3939
{% endfor %}
4040

4141
{% if language_rankings|length > 15 %}
42-
[View all {{ language_rankings|length }} languages...](All-Languages.md)
42+
[View all {{ language_rankings|length }} languages...](All-Languages)
4343
{% endif %}
4444

4545
---
@@ -64,7 +64,7 @@ Best translation quality by target language:
6464
| Language | Avg Score | Best Model |
6565
|----------|-----------|------------|
6666
{% for lang in category.languages %}
67-
| [{{ lang.name }}](languages/{{ lang.slug }}.md) | {{ lang.indicator }} {{ lang.avg_overall|round(1) }} | {{ lang.best_model }} |
67+
| [{{ lang.name }}](languages/{{ lang.slug }}) | {{ lang.indicator }} {{ lang.avg_overall|round(1) }} | {{ lang.best_model }} |
6868
{% endfor %}
6969

7070
{% endfor %}
@@ -73,10 +73,11 @@ Best translation quality by target language:
7373

7474
## Browse
7575

76-
- **By Language:** [All Languages](All-Languages.md)
77-
- **By Model:** [All Models](All-Models.md)
78-
- **Raw Data:** [Download JSON](https://github.com/yourusername/TranslateBookWithLLM/tree/main/benchmark_results)
76+
- **By Language:** [All Languages](All-Languages)
77+
- **By Model:** [All Models](All-Models)
78+
- **Benchmark Documentation:** [How to Run Benchmarks](Benchmark-System)
79+
- **Raw Data:** [Download JSON](https://github.com/hydropix/TranslateBookWithLLM/tree/main/benchmark_results)
7980

8081
---
8182

82-
*Generated by [TranslateBookWithLLM](https://github.com/yourusername/TranslateBookWithLLM) benchmark system*
83+
*Generated by [TranslateBookWithLLM](https://github.com/hydropix/TranslateBookWithLLM) benchmark system*

‎benchmark/wiki/templates/language.md.j2‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,8 @@
1515
| **Fluency** | {{ avg_fluency|round(1) }}/10 |
1616
| **Style** | {{ avg_style|round(1) }}/10 |
1717
| **Total Tests** | {{ total_translations }} |
18-
| **Best Model** | [{{ best_model }}](../models/{{ best_model_slug }}.md) |
19-
| **Worst Model** | [{{ worst_model }}](../models/{{ worst_model_slug }}.md) |
18+
| **Best Model** | [{{ best_model }}](../models/{{ best_model_slug }}) |
19+
| **Worst Model** | [{{ worst_model }}](../models/{{ worst_model_slug }}) |
2020

2121
---
2222

@@ -25,7 +25,7 @@
2525
| Model | Overall | Accuracy | Fluency | Style |
2626
|-------|---------|----------|---------|-------|
2727
{% for result in model_results %}
28-
| [{{ result.model }}](../models/{{ result.model_slug }}.md) | {{ result.indicator }} {{ result.avg_overall|round(1) }} | {{ result.avg_accuracy|round(1) }} | {{ result.avg_fluency|round(1) }} | {{ result.avg_style|round(1) }} |
28+
| [{{ result.model }}](../models/{{ result.model_slug }}) | {{ result.indicator }} {{ result.avg_overall|round(1) }} | {{ result.avg_accuracy|round(1) }} | {{ result.avg_fluency|round(1) }} | {{ result.avg_style|round(1) }} |
2929
{% endfor %}
3030

3131
---
@@ -70,4 +70,4 @@
7070

7171
---
7272

73-
[< Back to Home](../Home.md) | [All Languages](../All-Languages.md)
73+
[< Back to Home](../Home) | [All Languages](../All-Languages)

‎benchmark/wiki/templates/model.md.j2‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -14,8 +14,8 @@
1414
| **Style** | {{ avg_style|round(1) }}/10 |
1515
| **Languages Tested** | {{ total_languages }} |
1616
| **Total Translations** | {{ total_translations }} |
17-
| **Best Language** | [{{ best_language }}](../languages/{{ best_language_slug }}.md) ({{ best_language_score|round(1) }}) |
18-
| **Worst Language** | [{{ worst_language }}](../languages/{{ worst_language_slug }}.md) ({{ worst_language_score|round(1) }}) |
17+
| **Best Language** | [{{ best_language }}](../languages/{{ best_language_slug }}) ({{ best_language_score|round(1) }}) |
18+
| **Worst Language** | [{{ worst_language }}](../languages/{{ worst_language_slug }}) ({{ worst_language_score|round(1) }}) |
1919

2020
---
2121

@@ -26,7 +26,7 @@
2626
| Rank | Language | Overall | Accuracy | Fluency | Style |
2727
|------|----------|---------|----------|---------|-------|
2828
{% for lang in language_results[:10] %}
29-
| {{ loop.index }} | [{{ lang.name }}](../languages/{{ lang.slug }}.md) | {{ lang.indicator }} {{ lang.avg_overall|round(1) }} | {{ lang.avg_accuracy|round(1) }} | {{ lang.avg_fluency|round(1) }} | {{ lang.avg_style|round(1) }} |
29+
| {{ loop.index }} | [{{ lang.name }}](../languages/{{ lang.slug }}) | {{ lang.indicator }} {{ lang.avg_overall|round(1) }} | {{ lang.avg_accuracy|round(1) }} | {{ lang.avg_fluency|round(1) }} | {{ lang.avg_style|round(1) }} |
3030
{% endfor %}
3131

3232
{% if language_results|length > 10 %}
@@ -36,7 +36,7 @@
3636
| Rank | Language | Overall | Accuracy | Fluency | Style |
3737
|------|----------|---------|----------|---------|-------|
3838
{% for lang in language_results %}
39-
| {{ loop.index }} | [{{ lang.name }}](../languages/{{ lang.slug }}.md) | {{ lang.indicator }} {{ lang.avg_overall|round(1) }} | {{ lang.avg_accuracy|round(1) }} | {{ lang.avg_fluency|round(1) }} | {{ lang.avg_style|round(1) }} |
39+
| {{ loop.index }} | [{{ lang.name }}](../languages/{{ lang.slug }}) | {{ lang.indicator }} {{ lang.avg_overall|round(1) }} | {{ lang.avg_accuracy|round(1) }} | {{ lang.avg_fluency|round(1) }} | {{ lang.avg_style|round(1) }} |
4040
{% endfor %}
4141

4242
</details>
@@ -52,7 +52,7 @@
5252
| Language | Overall | Accuracy | Fluency | Style |
5353
|----------|---------|----------|---------|-------|
5454
{% for lang in category.languages %}
55-
| [{{ lang.name }}](../languages/{{ lang.slug }}.md) | {{ lang.indicator }} {{ lang.avg_overall|round(1) }} | {{ lang.avg_accuracy|round(1) }} | {{ lang.avg_fluency|round(1) }} | {{ lang.avg_style|round(1) }} |
55+
| [{{ lang.name }}](../languages/{{ lang.slug }}) | {{ lang.indicator }} {{ lang.avg_overall|round(1) }} | {{ lang.avg_accuracy|round(1) }} | {{ lang.avg_fluency|round(1) }} | {{ lang.avg_style|round(1) }} |
5656
{% endfor %}
5757

5858
**Category Average:** {{ category.indicator }} {{ category.avg_overall|round(1) }}
@@ -130,4 +130,4 @@
130130

131131
---
132132

133-
[< Back to Home](../Home.md) | [All Models](../All-Models.md)
133+
[< Back to Home](../Home) | [All Models](../All-Models)

0 commit comments

Comments
 (0)