You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
-**NLLB (No Language Left Behind)**: Supports 200+ languages, CC-BY-NC license
20
19
-**Language Detection**: Automatically detects the language of text using the Lingua language detector, which is highly accurate even for short text snippets
21
20
- Lingua prioritizes quality over quantity, focusing on accurate detection rather than supporting every possible language
22
21
- Supports 75 different languages with high precision, even though the translation models accept 200+ languages
The project includes several helpful make commands:
49
48
50
-
-`make install` - Install project dependencies
49
+
-`make serve` - Serves the API in port 8000
50
+
-`make download-model` - Downloads model from HuggingFace and stores locally in `./models`
51
+
-`make docker-up` - Serves the API and dockerized services like otel, jeager, prometheus, and valkey (cache)
52
+
53
+
### Development Commands
54
+
51
55
-`make test` - Run tests
52
56
-`make lint` - Run linters (flake8, isort, black)
53
57
-`make format` - Format code with isort and black
54
58
-`make coverage` - Run tests with coverage report
55
59
-`make coverage-html` - Generate HTML coverage report
56
60
57
-
## Testing and Code Coverage
58
-
59
-
### Running Tests with Coverage
60
-
61
-
To run tests with coverage reporting:
62
-
63
-
```bash
64
-
make coverage
65
-
```
66
-
67
-
For a detailed HTML coverage report:
68
-
69
-
```bash
70
-
make coverage-html
71
-
```
72
-
73
-
The HTML report will be generated in the `htmlcov` directory.
74
-
75
-
### Coverage Configuration
76
-
77
-
The project uses a `.coveragerc` file to configure coverage settings. This ensures consistent coverage reporting across different environments.
78
-
79
61
## Downloading Translation Models
80
62
81
63
Babeltron supports two types of translation models: M2M100 and NLLB (No Language Left Behind). You can download models of different sizes depending on your needs and hardware constraints:
@@ -318,6 +300,7 @@ The following environment variables can be used to configure the application:
318
300
-`PORT`: Port to run the API server on (default: `8000`)
319
301
-`WORKER_COUNT`: Number of worker processes to use (default: `1`)
320
302
-`BABELTRON_MODEL_TYPE`: Type of model to use in the API (`m2m100` or `nllb`, default: `m2m100`)
303
+
-`AUTH_USERNAME` and `AUTH_PASSWORD`: Enables basic auth on translate and detect endpoints
321
304
322
305
### Docker Volume Mounts
323
306
@@ -442,42 +425,12 @@ For a detailed technical overview of the system architecture, including diagrams
442
425
443
426

444
427
445
-
## Downloading Models
446
-
447
-
Before using Babeltron, you need to download at least one model:
448
-
449
-
### Translation Models
450
-
451
-
```bash
452
-
# Download the default M2M100 small model (418M parameters)
453
-
make download-model-m2m-small
454
-
455
-
# Download the M2M100 medium model (1.2B parameters)
456
-
make download-model-m2m-medium
457
-
458
-
# Download the M2M100 large model (12B parameters)
459
-
make download-model-m2m-large
460
-
461
-
# Download the NLLB small model (600M parameters)
462
-
make download-model-nllb-small
463
-
464
-
# Download the NLLB large model (3.3B parameters)
465
-
make download-model-nllb-large
466
-
```
467
-
468
-
### Language Detection Model
469
-
470
-
```bash
471
-
# Download the XLM-RoBERTa model for language detection
472
-
make download-model-xlm-roberta
473
-
```
474
-
475
428
## API Usage
476
429
477
430
### Translation
478
431
479
432
```bash
480
-
curl -X POST "http://localhost:8000/translate" \
433
+
curl -X POST "http://localhost:8000/api/v1/translate" \
481
434
-H "Content-Type: application/json" \
482
435
-d '{"text": "Hello, how are you?", "source_lang": "en", "target_lang": "fr"}'
483
436
```
@@ -489,10 +442,10 @@ Response:
489
442
}
490
443
```
491
444
492
-
### Language Detection
445
+
### Detection
493
446
494
447
```bash
495
-
curl -X POST "http://localhost:8000/detect" \
448
+
curl -X POST "http://localhost:8000/api/v1/detect" \
0 commit comments