Skip to content

Commit 004e3a0

Browse files
HolyMichaelHolyMichael
committed
documentation for nlp annotators
Signed-off-by: Miguel Brandão <[email protected]>
1 parent aaf0d10 commit 004e3a0

File tree

2 files changed

+478
-0
lines changed

2 files changed

+478
-0
lines changed

‎deepsearch/model/examples/dummy_nlp_annotator/README.MD‎

Lines changed: 168 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,168 @@
1+
# DummyNlpAnnotator
2+
## Introduction
3+
This is an example dummy NLP kind annotator it supports text data and annotates entities.
4+
5+
## Running the Annotator
6+
To run this example make sure you've installed the full environment including the optional installs provided in poetry
7+
8+
poetry install --all-extras
9+
10+
Then simply start the server with
11+
12+
python -m deepsearch.model.examples.dummy_nlp_annotator.main
13+
14+
## Simple Interaction with the Annotator
15+
16+
You can direcly access the API via a browser to the provided url on the console upon running the application, usually:
17+
18+
http://127.0.0.1:8000
19+
This will take you to the landing page. Here you will likely find that you are not authenticated, however you can still check if the API is responsive by accessing the /health endpoint
20+
21+
http://127.0.0.1:8000/health
22+
It will be easier to interact with the application via the provided documentation endpoint
23+
24+
http://127.0.0.1:8000/docs
25+
26+
## Security
27+
By default, the API requires an API-key to be used with every request to most endpoints, this key is defined on:
28+
29+
deepsearch/model/examples/dummy_nlp_annotator/main.py
30+
this API key must be provided on the authorization header, sample request headers to /:
31+
32+
{'host': '127.0.0.1:8000', 'connection': 'keep-alive', 'sec-ch-ua': '"Not.A/Brand";v="8", "Chromium";v="114", "Google Chrome";v="114"', 'accept': 'application/json', 'sec-ch-ua-mobile': '?0', 'authorization': 'example123', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36', 'sec-ch-ua-platform': '"Linux"', 'sec-fetch-site': 'same-origin', 'sec-fetch-mode': 'cors', 'sec-fetch-dest': 'empty', 'referer': 'http://127.0.0.1:8000/docs', 'accept-encoding': 'gzip, deflate, br', 'accept-language': 'en-US,en;q=0.9'}
33+
34+
## Advanced Interaction with the Annotator
35+
On the /docs endpoint after inserting the api key you may see the following information about the API server
36+
37+
on endpoint:
38+
39+
- / - A list of all the annotators hosted on this server, in this example you will find only "DummyNLPAnnotator" on each annotator you will find its annotation capabilities as well as the kind of annotator it is (NLPAnnotator) which in turn tells you how to make requests to the annotator
40+
- /model/{model_name} - You will find the annotation capabilities for the given annotator as well as it's kind.
41+
- /model/{model_name}/predict - You can make POST requests to have the model annotate your data, refer to [Sample Requests](#Sample-Requests)
42+
43+
## Sample Requests
44+
45+
```python
46+
{
47+
"apiVersion": "string",
48+
"kind": "NLPModel",
49+
"metadata": {
50+
"annotations": {
51+
"deepsearch.res.ibm.com/x-deadline": "2038-01-18T00:00:00.000Z",
52+
"deepsearch.res.ibm.com/x-transaction-id": "string",
53+
"deepsearch.res.ibm.com/x-attempt-number": "string",
54+
"deepsearch.res.ibm.com/x-max-attempts": "string"
55+
}
56+
},
57+
"spec": {
58+
"findEntities": {
59+
"entityNames": ["entity_foo", "entity_bar"],
60+
"objectType": "text",
61+
"texts": [
62+
"A piece of text",
63+
"Yet another piece of text"
64+
]
65+
}
66+
}
67+
}
68+
```
69+
70+
- You may alter entityNames to have any number of the entity types the annotator declares it can annotate, or an empty list to annotate all.
71+
- This annotator has declared that it can only annotate text, as such the objectType must be text
72+
- texts may be as long or as short as you need it.
73+
- The x-deadline must lie some time in the future
74+
- This annotator has declared that it is of kind NLPModel as such the kind for the request must match
75+
- refer to the /docs for details on the NLPRequest type
76+
77+
Will result in the following output:
78+
79+
```python
80+
{
81+
"entities":[
82+
{
83+
"entity_foo":[
84+
{
85+
"type":"entity_foo",
86+
"match":"a 'entity_foo' match in 'A piece of text'",
87+
"original":"a 'entity_foo' original in 'A piece of text'",
88+
"range":[
89+
1,
90+
5
91+
]
92+
},
93+
{
94+
"type":"entity_foo",
95+
"match":"another 'entity_foo' match in 'A piece of text'",
96+
"original":"another 'entity_foo' original in 'A piece of text'",
97+
"range":[
98+
12,
99+
42
100+
]
101+
}
102+
],
103+
"entity_bar":[
104+
{
105+
"type":"entity_bar",
106+
"match":"a 'entity_bar' match in 'A piece of text'",
107+
"original":"a 'entity_bar' original in 'A piece of text'",
108+
"range":[
109+
1,
110+
5
111+
]
112+
},
113+
{
114+
"type":"entity_bar",
115+
"match":"another 'entity_bar' match in 'A piece of text'",
116+
"original":"another 'entity_bar' original in 'A piece of text'",
117+
"range":[
118+
12,
119+
42
120+
]
121+
}
122+
]
123+
},
124+
{
125+
"entity_foo":[
126+
{
127+
"type":"entity_foo",
128+
"match":"a 'entity_foo' match in 'Yet another piece of text'",
129+
"original":"a 'entity_foo' original in 'Yet another piece of text'",
130+
"range":[
131+
1,
132+
5
133+
]
134+
},
135+
{
136+
"type":"entity_foo",
137+
"match":"another 'entity_foo' match in 'Yet another piece of text'",
138+
"original":"another 'entity_foo' original in 'Yet another piece of text'",
139+
"range":[
140+
12,
141+
42
142+
]
143+
}
144+
],
145+
"entity_bar":[
146+
{
147+
"type":"entity_bar",
148+
"match":"a 'entity_bar' match in 'Yet another piece of text'",
149+
"original":"a 'entity_bar' original in 'Yet another piece of text'",
150+
"range":[
151+
1,
152+
5
153+
]
154+
},
155+
{
156+
"type":"entity_bar",
157+
"match":"another 'entity_bar' match in 'Yet another piece of text'",
158+
"original":"another 'entity_bar' original in 'Yet another piece of text'",
159+
"range":[
160+
12,
161+
42
162+
]
163+
}
164+
]
165+
}
166+
]
167+
}
168+
```

0 commit comments

Comments
 (0)