|
1 | | -# Model API |
2 | | -> Currently in **beta**. |
3 | | -
|
4 | | -The Model API is a unified and extensible inference API across different model kinds. |
5 | | - |
6 | | -Built-in model kind support includes NLP annotators and QA generators. |
7 | | - |
8 | | -## Installation |
9 | | -To use the Model API, install including the `api` extra, i.e.: |
10 | | -- with poetry: |
11 | | -`poetry add "deepsearch-toolkit[api]"` |
12 | | -- with pip: `pip install "deepsearch-toolkit[api]"` |
13 | | - |
14 | | -## Basic usage |
15 | | -```python |
16 | | -from deepsearch.model.server.config import Settings |
17 | | -from deepsearch.model.server.model_app import ModelApp |
18 | | - |
19 | | -# (1) create an app |
20 | | -app = ModelApp(settings=Settings()) |
21 | | - |
22 | | -# (2) register your model(s) |
23 | | -model = ... # e.g. SimpleGeoNLPAnnotator() |
24 | | -app.register_model(model) |
25 | | - |
26 | | -# (3) run the app |
27 | | -app.run(host="127.0.0.1", port=8000) |
28 | | -``` |
29 | | - |
30 | | -### Settings |
31 | | -App configuration is done in [`Settings`](server/config.py) based on |
32 | | -[Pydantic Settings with dotenv support](https://docs.pydantic.dev/dev-v1/usage/settings/). |
33 | | - |
34 | | -E.g. the required API key can be injected via env var `DS_MODEL_API_KEY`. |
35 | | - |
36 | | -### OpenAPI |
37 | | -The OpenAPI UI is served under `/docs`, i.e. by default at http://127.0.0.1:8000/docs. |
38 | | - |
39 | | -## Developing a new model |
40 | | -To develop a new model class for an existing [kind](kinds/), inherit from the base model |
41 | | -class of that kind and implement the abstract methods and attributes. |
42 | | - |
43 | | -The framework will automatically use the correct controller for your model. |
44 | | - |
45 | | -To use a custom controller instead, pass it to `ModelApp.register_model()` via the |
46 | | -optional parameter `controller`. |
47 | | - |
48 | | -### Examples |
49 | | -- [Dummy NLP annotator](examples/dummy_nlp_annotator/) |
50 | | -- [Simple geo NLP annotator](examples/simple_geo_nlp_annotator/) |
51 | | -- [Dummy QA generator](examples/dummy_qa_generator/) |
52 | | - |
53 | | -Note: these examples configure the app with API key "example123"; when running them, use |
54 | | -the same key to access the protected endpoints. |
55 | | - |
56 | | -### Inference |
57 | | -As as example, an input payload for the `/predict` endpoint for the geography annotator |
58 | | -could look as follows (note that `deepsearch.res.ibm.com/x-deadline` should be a |
59 | | -future timestamp): |
60 | | -```json |
61 | | -{ |
62 | | - "apiVersion": "v1", |
63 | | - "kind": "NLPModel", |
64 | | - "metadata": { |
65 | | - "annotations": { |
66 | | - "deepsearch.res.ibm.com/x-deadline": "2024-04-20T12:26:01.479484+00:00", |
67 | | - "deepsearch.res.ibm.com/x-transaction-id": "abc", |
68 | | - "deepsearch.res.ibm.com/x-attempt-number": 5, |
69 | | - "deepsearch.res.ibm.com/x-max-attempts": 5 |
70 | | - } |
71 | | - }, |
72 | | - "spec": { |
73 | | - "findEntities": { |
74 | | - "entityNames": ["cities", "countries"], |
75 | | - "objectType": "text", |
76 | | - "texts": [ |
77 | | - "Bern, the capital city of Switzerland, is built around a crook in the Aare River.", |
78 | | - "Athens is a major coastal urban area in the Mediterranean and is both the capital and largest city of Greece." |
79 | | - ] |
80 | | - } |
81 | | - } |
82 | | -} |
83 | | -``` |
0 commit comments