Skip to content

Commit f5bd12c

Browse files
angptangpt
committed
Adding docs for the CLI
1 parent a0aa6fb commit f5bd12c

File tree

2 files changed

+283
-0
lines changed

2 files changed

+283
-0
lines changed

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,8 @@ target/
8787
profile_default/
8888
ipython_config.py
8989

90+
**/.DS_Store
91+
9092
# pyenv
9193
# For a library or package, you might want to ignore these files since the code is
9294
# intended to run in multiple environments; otherwise, check them in:

‎docs/cli.md‎

Lines changed: 281 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,281 @@
1+
# Encoderfile CLI Documentation
2+
3+
## Overview
4+
5+
Encoderfile is a command-line tool for running machine learning inference tasks using transformer models. It supports multiple model types including embeddings, sequence classification, and token classification. The tool can operate in two modes: as a server (HTTP/gRPC) or as a standalone inference tool.
6+
7+
## Architecture
8+
9+
The CLI is built with the following components:
10+
- **Server Mode**: Hosts models via HTTP and/or gRPC endpoints
11+
- **Inference Mode**: Performs one-off inference operations from the command line
12+
- **Multi-Model Support**: Automatically detects and routes to the appropriate model type
13+
14+
## Commands
15+
16+
### `serve`
17+
18+
Starts the encoderfile server with HTTP and/or gRPC endpoints for model inference.
19+
20+
#### Usage
21+
22+
```bash
23+
encoderfile serve [OPTIONS]
24+
```
25+
26+
#### Options
27+
28+
| Option | Type | Default | Description |
29+
|--------|------|---------|-------------|
30+
| `--grpc-hostname` | String | `[::]` | Hostname/IP address for the gRPC server |
31+
| `--grpc-port` | String | `50051` | Port for the gRPC server |
32+
| `--http-hostname` | String | `0.0.0.0` | Hostname/IP address for the HTTP server |
33+
| `--http-port` | String | `8080` | Port for the HTTP server |
34+
| `--disable-grpc` | Boolean | `false` | Disable the gRPC server |
35+
| `--disable-http` | Boolean | `false` | Disable the HTTP server |
36+
37+
#### Examples
38+
39+
**Start both HTTP and gRPC servers (default):**
40+
```bash
41+
encoderfile serve
42+
```
43+
44+
**Start only HTTP server:**
45+
```bash
46+
encoderfile serve --disable-grpc
47+
```
48+
49+
**Start only gRPC server:**
50+
```bash
51+
encoderfile serve --disable-http
52+
```
53+
54+
**Custom ports and hostnames:**
55+
```bash
56+
encoderfile serve \
57+
--http-hostname 127.0.0.1 \
58+
--http-port 3000 \
59+
--grpc-hostname localhost \
60+
--grpc-port 50052
61+
```
62+
63+
#### Notes
64+
65+
- At least one server type (HTTP or gRPC) must be enabled
66+
- The server will display a banner upon successful startup
67+
- Both servers run concurrently using async tasks
68+
69+
---
70+
71+
### `infer`
72+
73+
Performs inference on input text using the configured model. The model type is automatically detected based on configuration.
74+
75+
#### Usage
76+
77+
```bash
78+
encoderfile infer <INPUTS>... [OPTIONS]
79+
```
80+
81+
#### Arguments
82+
83+
| Argument | Required | Description |
84+
|----------|----------|-------------|
85+
| `<INPUTS>` | Yes | One or more text strings to process |
86+
87+
#### Options
88+
89+
| Option | Type | Default | Description |
90+
|--------|------|---------|-------------|
91+
| `--normalize` | Boolean | `true` | Normalize embeddings (embedding models only) |
92+
| `-f, --format` | Enum | `json` | Output format (currently only JSON supported) |
93+
| `-o, --out-dir` | String | None | Output file path; if not provided, prints to stdout |
94+
95+
#### Model Types
96+
97+
The inference behavior depends on the model type configured:
98+
99+
##### 1. Embedding Models
100+
Generates vector embeddings for input text.
101+
102+
**Example:**
103+
```bash
104+
encoderfile infer "Hello world" "Another sentence"
105+
```
106+
107+
**With normalization disabled:**
108+
```bash
109+
encoderfile infer "Hello world" --normalize=false
110+
```
111+
112+
##### 2. Sequence Classification Models
113+
Classifies entire sequences (e.g., sentiment analysis, topic classification).
114+
115+
**Example:**
116+
```bash
117+
encoderfile infer "This product is amazing!" "I'm very disappointed"
118+
```
119+
120+
##### 3. Token Classification Models
121+
Labels individual tokens (e.g., Named Entity Recognition, Part-of-Speech tagging).
122+
123+
**Example:**
124+
```bash
125+
encoderfile infer "Apple Inc. is located in Cupertino, California"
126+
```
127+
128+
#### Output Formats
129+
130+
Currently, only JSON format is supported (`--format json`). The output structure varies by model type:
131+
132+
##### Embedding Output
133+
```json
134+
{
135+
"embeddings": [
136+
[0.123, -0.456, 0.789, ...],
137+
[0.321, -0.654, 0.987, ...]
138+
],
139+
"metadata": null
140+
}
141+
```
142+
143+
##### Sequence Classification Output
144+
```json
145+
{
146+
"predictions": [
147+
{
148+
"label": "POSITIVE",
149+
"score": 0.9876
150+
},
151+
{
152+
"label": "NEGATIVE",
153+
"score": 0.8765
154+
}
155+
],
156+
"metadata": null
157+
}
158+
```
159+
160+
##### Token Classification Output
161+
```json
162+
{
163+
"predictions": [
164+
{
165+
"tokens": ["Apple", "Inc.", "is", "located", "in", "Cupertino", ",", "California"],
166+
"labels": ["B-ORG", "I-ORG", "O", "O", "O", "B-LOC", "O", "B-LOC"]
167+
}
168+
],
169+
"metadata": null
170+
}
171+
```
172+
173+
#### Saving Output to File
174+
175+
**Save results to a file:**
176+
```bash
177+
encoderfile infer "Sample text" -o results.json
178+
```
179+
180+
**Process multiple inputs and save:**
181+
```bash
182+
encoderfile infer "First input" "Second input" "Third input" --out-dir output.json
183+
```
184+
185+
## Configuration
186+
187+
The CLI relies on external configuration to determine:
188+
- Model type (Embedding, SequenceClassification, TokenClassification)
189+
- Model path and parameters
190+
- Server settings
191+
192+
Ensure your configuration is properly set before running commands. Refer to the main encoderfile configuration documentation for details.
193+
194+
## Error Handling
195+
196+
The CLI will return appropriate error messages for:
197+
- Invalid configuration (e.g., both servers disabled)
198+
- Missing required arguments
199+
- Model loading failures
200+
- Inference errors
201+
- File I/O errors
202+
203+
## Examples
204+
205+
### Basic Inference Workflow
206+
207+
```bash
208+
# Set up configuration (example)
209+
export MODEL_PATH=/path/to/model
210+
export MODEL_TYPE=embedding
211+
212+
# Run inference
213+
encoderfile infer "Hello world"
214+
215+
# Save to file
216+
encoderfile infer "Hello world" -o embedding.json
217+
```
218+
219+
### Server Workflow
220+
221+
```bash
222+
# Terminal 1: Start server
223+
encoderfile serve --http-port 8080
224+
225+
# Terminal 2: Make HTTP requests (using curl)
226+
curl -X POST http://localhost:8080/infer \
227+
-H "Content-Type: application/json" \
228+
-d '{"inputs": ["Hello world"], "normalize": true}'
229+
```
230+
231+
### Batch Processing
232+
233+
```bash
234+
# Process multiple inputs at once
235+
encoderfile infer \
236+
"First document to analyze" \
237+
"Second document to analyze" \
238+
"Third document to analyze" \
239+
--out-dir batch_results.json
240+
```
241+
242+
### Custom Server Configuration
243+
244+
```bash
245+
# Run on specific network interface with custom ports
246+
encoderfile serve \
247+
--http-hostname 192.168.1.100 \
248+
--http-port 3000 \
249+
--grpc-hostname 192.168.1.100 \
250+
--grpc-port 50052
251+
```
252+
253+
## Troubleshooting
254+
255+
### Both servers cannot be disabled
256+
**Error**: "Cannot disable both gRPC and HTTP"
257+
258+
**Solution**: Enable at least one server type:
259+
```bash
260+
encoderfile serve --disable-grpc # Keep HTTP enabled
261+
# OR
262+
encoderfile serve --disable-http # Keep gRPC enabled
263+
```
264+
265+
### Output not appearing
266+
If output isn't visible, check:
267+
1. Ensure you're not redirecting output to a file unintentionally
268+
2. Check file permissions if using `--out-dir`
269+
3. Verify the model is correctly configured
270+
271+
### Model type detection
272+
The CLI automatically detects model type from configuration. If inference behaves unexpectedly:
273+
1. Verify your model configuration
274+
2. Ensure the model type matches your use case
275+
3. Check model compatibility
276+
277+
## Additional Resources
278+
279+
- GitHub Repository: [mozilla-ai/encoderfile](https://github.com/mozilla-ai/encoderfile)
280+
- API Documentation: Refer to the server API docs for HTTP/gRPC endpoint specifications
281+
- Model Configuration: See configuration documentation for model setup

0 commit comments

Comments
 (0)