Skip to content

Commit d9c4fda

Browse files
Making the index reference doc up to date. (#1068)
* Making the index reference doc up to date. We recently integrated index-id and index-uri in the index-reference. In addition this PR makes more minor change like adding a source in the example configuration. * Update docs/reference/index-config.md Co-authored-by: Evance Soumaoro <[email protected]> Co-authored-by: Evance Soumaoro <[email protected]>
1 parent 1946c12 commit d9c4fda

File tree

1 file changed

+34
-2
lines changed

1 file changed

+34
-2
lines changed

docs/reference/index-config.md

Lines changed: 34 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,9 @@ position: 4
55

66
This page describes how to configure an index.
77

8-
The index configuration lets you define four items:
8+
In addition to the `index_id`, the index configuration lets you define five items:
99

10+
- The **index-uri**: it defines where the index files should be stored.
1011
- The **doc mapping**: it defines how a document and the fields it contains are stored and indexed for a given index.
1112
- The **indexing settings**: it defines the timestamp field used for sharding, and some more advanced parameters like the merge policy.
1213
- The **search settings**: it defines the default search fields `default_search_fields`, a list of fields that Quickwit will search into if the user query does not explicitly target a field.
@@ -19,8 +20,13 @@ Configuration is set at index creation and cannot be modified except for the sou
1920
The index configuration format is YAML. When a key is absent from the configuration file, the default value is used.
2021
Here is a complete example suited for the HDFS logs dataset:
2122

22-
```
23+
```yaml
2324
version: 0 # File format version.
25+
26+
index_id: "hdfs"
27+
28+
index_uri: "s3://my-bucket/hdfs"
29+
2430
doc_mapping:
2531
field_mappings:
2632
- name: timestamp
@@ -40,13 +46,39 @@ doc_mapping:
4046
type: text
4147
tokenizer: raw
4248
tag_fields: ["resource.service"]
49+
4350
indexing_settings:
4451
timestamp_field: timestamp
52+
4553
search_settings:
4654
default_search_fields: [severity_text, body]
4755

56+
sources:
57+
- hdfs: hdfs-log-kafka
58+
source_type: kafka
59+
params:
60+
topic: hdfs-logs
61+
client_params:
62+
bootstrap.servers: localhost:9092
63+
group.id: quickwit-consumer-group
64+
security.protocol: SSL
4865
```
4966
67+
## Index uri
68+
69+
The index-uri defines where the index files (also called splits) should be stored.
70+
This parameter expects a [storage uri](storage-uri).
71+
72+
73+
The `index-uri` parameter is optional.
74+
By default, the `index-uri` will be computed by concatenating the `index-id` with the
75+
`default_index_root_uri` defined in the [Quickwit's config](quickwit-config).
76+
77+
<aside>
78+
⚠️ The file storage will not work when running quickwit in distributed mode.
79+
Today, only the s3 storage is available when running several searcher nodes.
80+
</aside>
81+
5082
## Doc mapping
5183

5284
The doc mapping defines how a document and the fields it contains are stored and indexed for a given index. A document is a collection of named fields, each having its own data type (text, binary, date, i64, u64, f64).

0 commit comments

Comments
 (0)