Skip to content

Commit 1bcb73e

Browse files
committed
revised PR
Signed-off-by: Miguel Brandão <[email protected]>
1 parent fd8e0ee commit 1bcb73e

File tree

7 files changed

+331
-404
lines changed

7 files changed

+331
-404
lines changed

deepsearch/model/README.md

Lines changed: 0 additions & 83 deletions
Original file line numberDiff line numberDiff line change
@@ -1,83 +0,0 @@
1-
# Model API
2-
> Currently in **beta**.
3-
4-
The Model API is a unified and extensible inference API across different model kinds.
5-
6-
Built-in model kind support includes NLP annotators and QA generators.
7-
8-
## Installation
9-
To use the Model API, install including the `api` extra, i.e.:
10-
- with poetry:
11-
`poetry add "deepsearch-toolkit[api]"`
12-
- with pip: `pip install "deepsearch-toolkit[api]"`
13-
14-
## Basic usage
15-
```python
16-
from deepsearch.model.server.config import Settings
17-
from deepsearch.model.server.model_app import ModelApp
18-
19-
# (1) create an app
20-
app = ModelApp(settings=Settings())
21-
22-
# (2) register your model(s)
23-
model = ... # e.g. SimpleGeoNLPAnnotator()
24-
app.register_model(model)
25-
26-
# (3) run the app
27-
app.run(host="127.0.0.1", port=8000)
28-
```
29-
30-
### Settings
31-
App configuration is done in [`Settings`](server/config.py) based on
32-
[Pydantic Settings with dotenv support](https://docs.pydantic.dev/dev-v1/usage/settings/).
33-
34-
E.g. the required API key can be injected via env var `DS_MODEL_API_KEY`.
35-
36-
### OpenAPI
37-
The OpenAPI UI is served under `/docs`, i.e. by default at http://127.0.0.1:8000/docs.
38-
39-
## Developing a new model
40-
To develop a new model class for an existing [kind](kinds/), inherit from the base model
41-
class of that kind and implement the abstract methods and attributes.
42-
43-
The framework will automatically use the correct controller for your model.
44-
45-
To use a custom controller instead, pass it to `ModelApp.register_model()` via the
46-
optional parameter `controller`.
47-
48-
### Examples
49-
- [Dummy NLP annotator](examples/dummy_nlp_annotator/)
50-
- [Simple geo NLP annotator](examples/simple_geo_nlp_annotator/)
51-
- [Dummy QA generator](examples/dummy_qa_generator/)
52-
53-
Note: these examples configure the app with API key "example123"; when running them, use
54-
the same key to access the protected endpoints.
55-
56-
### Inference
57-
As as example, an input payload for the `/predict` endpoint for the geography annotator
58-
could look as follows (note that `deepsearch.res.ibm.com/x-deadline` should be a
59-
future timestamp):
60-
```json
61-
{
62-
"apiVersion": "v1",
63-
"kind": "NLPModel",
64-
"metadata": {
65-
"annotations": {
66-
"deepsearch.res.ibm.com/x-deadline": "2024-04-20T12:26:01.479484+00:00",
67-
"deepsearch.res.ibm.com/x-transaction-id": "abc",
68-
"deepsearch.res.ibm.com/x-attempt-number": 5,
69-
"deepsearch.res.ibm.com/x-max-attempts": 5
70-
}
71-
},
72-
"spec": {
73-
"findEntities": {
74-
"entityNames": ["cities", "countries"],
75-
"objectType": "text",
76-
"texts": [
77-
"Bern, the capital city of Switzerland, is built around a crook in the Aare River.",
78-
"Athens is a major coastal urban area in the Mediterranean and is both the capital and largest city of Greece."
79-
]
80-
}
81-
}
82-
}
83-
```
File renamed without changes.
File renamed without changes.
File renamed without changes.

docs/guide/index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,4 +18,4 @@
1818
- [Custom CLI plugins](./cli_plugins.md)
1919

2020
## Custom models
21-
- [Custom model examples](./model.md)
21+
- [Model API](./model.md)

0 commit comments

Comments
 (0)