Skip to content

Commit 564a556

Browse files
committed
doc: update README
Signed-off-by: Xin Liu <sam@secondstate.io>
1 parent 95618cb commit 564a556

File tree

1 file changed

+203
-0
lines changed

1 file changed

+203
-0
lines changed

README.md

Lines changed: 203 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,203 @@
1+
# Keyword Search Server
2+
3+
kw-search-server is a web server that provides keyword search service. It provides the following features:
4+
5+
- Indexing documents via the `/v1/index/create` endpoint
6+
- Keyword search via the `/v1/search` endpoint
7+
- Download index file via the `/v1/index/download/{index_name}` endpoint
8+
9+
> [!IMPORTANT]
10+
> This project is still in the active development stage.
11+
12+
## Quick Start
13+
14+
- Download `kw-search-server` binary
15+
16+
```bash
17+
export VERSION=0.1.0
18+
19+
# macOS on Apple Silicon
20+
curl -LO https://github.com/LlamaEdge/kw-search-server/releases/download/${VERSION}/server-assistant-aarch64-apple-darwin.tar.gz
21+
tar -xvzf server-assistant-aarch64-apple-darwin.tar.gz
22+
23+
# macOS on Intel
24+
curl -LO https://github.com/LlamaEdge/kw-search-server/releases/download/${VERSION}/server-assistant-x86_64-apple-darwin.tar.gz
25+
tar -xvzf server-assistant-x86_64-apple-darwin.tar.gz
26+
27+
# Linux
28+
curl -LO https://github.com/LlamaEdge/kw-search-server/releases/download/${VERSION}/server-assistant-x86_64-unknown-linux-gnu.tar.gz
29+
tar -xvzf server-assistant-x86_64-unknown-linux-gnu.tar.gz
30+
```
31+
32+
- Run `kw-search-server`
33+
34+
```bash
35+
# Run server on default port 9069
36+
./kw-search-server
37+
38+
# Run server on custom port, e.g. 10086
39+
./kw-search-server --port 10086
40+
41+
# Or, run server with custom socket address, e.g. 0.0.0.0:10086
42+
./kw-search-server --socket-addr 0.0.0.0:10086
43+
```
44+
45+
To see all CLI options:
46+
47+
```bash
48+
./kw-search-server --help
49+
50+
Usage: kw-search-server [OPTIONS]
51+
52+
Options:
53+
--download-url-prefix <DOWNLOAD_URL_PREFIX>
54+
Download URL prefix, format: `http(s)://{IPv4_address}:{port}` or `http(s)://{domain}:{port}`
55+
--socket-addr <SOCKET_ADDR>
56+
Socket address of llama-proxy-server instance. For example, `0.0.0.0:9069`
57+
--port <PORT>
58+
Socket address of llama-proxy-server instance [default: 9069]
59+
-h, --help
60+
Print help
61+
-V, --version
62+
Print version
63+
```
64+
65+
## Usage: indexing and keyword search
66+
67+
### Create index
68+
69+
To create an index for a list of documents, you can use the `/v1/index/create` endpoint.
70+
71+
- Index for a list of documents
72+
73+
```bash
74+
# Create index from a list of files
75+
curl --location 'http://localhost:9069/v1/index/create' \
76+
--form 'file1=@"paris.txt"' \
77+
--form 'file2=@"paris.md"'
78+
```
79+
80+
If the index is created successfully, the response body in JSON format is as follows:
81+
82+
```json
83+
{
84+
"results": [
85+
{
86+
"filename": "paris.txt",
87+
"status": "indexed",
88+
"error": null
89+
},
90+
{
91+
"filename": "paris.md",
92+
"status": "indexed",
93+
"error": null
94+
}
95+
],
96+
"index_name": "index-4aae1cf6-d8dc-4233-b2bb-43911f9b74fd",
97+
"download_url": "http://localhost:9069/v1/index/download/index-4aae1cf6-d8dc-4233-b2bb-43911f9b74fd"
98+
}
99+
```
100+
101+
**Note** that the files should be of `txt` or `md` format.
102+
103+
- Index for a list of chunks
104+
105+
```bash
106+
# Create index from a list of chunks
107+
curl --location 'http://localhost:9069/v1/index/create' \
108+
--header 'Content-Type: application/json' \
109+
--data '{
110+
"documents": [
111+
{
112+
"content": "Paris, city and capital of France, ..."
113+
"title": "section 1" # optional
114+
},
115+
{
116+
"content": "Paris occupies a central position ..."
117+
"title": "section 2" # optional
118+
},
119+
{
120+
"content": "For centuries Paris has been one ..."
121+
"title": "section 3" # optional
122+
},
123+
{
124+
"content": "Paris’s site at a crossroads ..."
125+
"title": "section 4" # optional
126+
}
127+
]
128+
}'
129+
```
130+
131+
If the index is created successfully, the response body in JSON format is as follows:
132+
133+
```json
134+
{
135+
"results": [
136+
{
137+
"filename": "section 1",
138+
"status": "indexed",
139+
"error": null
140+
},
141+
{
142+
"filename": "section 2",
143+
"status": "indexed",
144+
"error": null
145+
},
146+
{
147+
"filename": "section 3",
148+
"status": "indexed",
149+
"error": null
150+
},
151+
{
152+
"filename": "section 4",
153+
"status": "indexed",
154+
"error": null
155+
}
156+
],
157+
"index_name": "index-a1f79ad1-b47b-4e36-948e-a591646ca014",
158+
"download_url": "http://localhost:9069/v1/index/download/index-a1f79ad1-b47b-4e36-948e-a591646ca014"
159+
}
160+
```
161+
162+
### Perform keyword search
163+
164+
To perform a keyword search, you can use the `/v1/search` endpoint:
165+
166+
```bash
167+
curl --location 'http://localhost:9069/v1/search' \
168+
--header 'Content-Type: application/json' \
169+
--data '{
170+
"query": "What is the location of Paris, France along the Seine river?",
171+
"top_k": 5,
172+
"index": "index-a1f79ad1-b47b-4e36-948e-a591646ca014"
173+
}'
174+
```
175+
176+
If the search is successful, the response body in JSON format is as follows:
177+
178+
```json
179+
{
180+
"hits": [
181+
{
182+
"title": "section 1",
183+
"content": "Paris, city and capital of France, situated in the north-central part of the country. People were living on the site of the present-day city, located along the Seine River some 233 miles (375 km) upstream from the river’s mouth on the English Channel (La Manche), by about 7600 BCE. The modern city has spread from the island (the Île de la Cité) and far beyond both banks of the Seine.",
184+
"score": 5.5615416
185+
},
186+
{
187+
"title": "section 2",
188+
"content": "Paris occupies a central position in the rich agricultural region known as the Paris Basin, and it constitutes one of eight départements of the Île-de-France administrative region. It is by far the country’s most important centre of commerce and culture. Area city, 41 square miles (105 square km); metropolitan area, 890 square miles (2,300 square km).",
189+
"score": 1.0718238
190+
},
191+
{
192+
"title": "section 3",
193+
"content": "Paris’s site at a crossroads of both water and land routes significant not only to France but also to Europe has had a continuing influence on its growth. Under Roman administration, in the 1st century BCE, the original site on the Île de la Cité was designated the capital of the Parisii tribe and territory. The Frankish king Clovis I had taken Paris from the Gauls by 494 CE and later made his capital there.",
194+
"score": 1.0254539
195+
},
196+
{
197+
"title": "section 4",
198+
"content": "For centuries Paris has been one of the world’s most important and attractive cities. It is appreciated for the opportunities it offers for business and commerce, for study, for culture, and for entertainment; its gastronomy, haute couture, painting, literature, and intellectual community especially enjoy an enviable reputation. Its sobriquet “the City of Light” (“la Ville Lumière”), earned during the Enlightenment, remains appropriate, for Paris has retained its importance as a centre for education and intellectual pursuits.",
199+
"score": 0.6374644
200+
}
201+
]
202+
}
203+
```

0 commit comments

Comments
 (0)