Skip to content

Commit 30f7fd9

Browse files
renetapopovamauvo
renetapopova
and
mauvo
authored
Dev memory config vector indexes (#2289) (#2320)
Co-authored-by: David Pond <[email protected]>
1 parent 77c8921 commit 30f7fd9

File tree

4 files changed

+168
-0
lines changed

4 files changed

+168
-0
lines changed

‎modules/ROOT/content-nav.adoc‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -213,6 +213,7 @@
213213
214214
* xref:performance/index.adoc[]
215215
** xref:performance/memory-configuration.adoc[]
216+
** xref:performance/vector-index-memory-configuration.adoc[]
216217
** xref:performance/index-configuration.adoc[]
217218
** xref:performance/gc-tuning.adoc[]
218219
** xref:performance/bolt-thread-pool-configuration.adoc[]

‎modules/ROOT/pages/performance/index.adoc‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ This section describes factors that affect operational performance and how to tu
66
The following topics are covered:
77

88
* xref:performance/memory-configuration.adoc[Memory configuration] -- How to configure memory settings for efficient operations.
9+
* xref:performance/vector-index-memory-configuration.adoc[Vector index memory configuration] -- How to configure memory for vector indexes.
910
* xref:performance/index-configuration.adoc[Index configuration] -- How to configure indexes.
1011
* xref:performance/gc-tuning.adoc[Garbage collector] -- How to configure the Java Virtual Machine's garbage collector.
1112
* xref:performance/bolt-thread-pool-configuration.adoc[Bolt thread pool configuration] -- How to configure the Bolt thread pool.

‎modules/ROOT/pages/performance/memory-configuration.adoc‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,9 @@ It is not possible to explicitly configure the amount of RAM that should be rese
1515
1GB is a good starting point for a server that is dedicated to running Neo4j.
1616
However, there are cases where the amount reserved for the OS is significantly larger than 1GB, such as servers with exceptionally large RAM.
1717
+
18+
If you have a vector index, you need to ensure that the OS has sufficient memory set aside for the vector index to perform optimally, because the vector index is loaded in OS memory and not in Neo4j page cache.
19+
For more information, see xref:performance/vector-index-memory-configuration.adoc[Vector index memory configuration].
20+
+
1821
[NOTE]
1922
====
2023
If you do not leave enough space for the OS, it will start to swap memory to disk, which will heavily affect performance.

‎modules/ROOT/pages/performance/vector-index-memory-configuration.adoc‎

Lines changed: 163 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,163 @@
1+
= Vector index memory configuration
2+
:description: How to configure Neo4j vector indexes to enhance performance in search operations.
3+
4+
link:{neo4j-docs-base-uri}/cypher-manual/current/indexes/semantic-indexes/vector-indexes/[Vector indexes] are based on link:https://lucene.apache.org/[Lucene].
5+
Lucene does not use Neo4j page cache memory, it uses OS memory as described in the xref:performance/memory-configuration.adoc[Memory configuration] section.
6+
When you have a vector index, you must ensure that there is sufficient memory for the JVM heap, Neo4j page cache, and the Lucene vector indexes to perform optimally.
7+
If Lucene has insufficient memory, the OS will perform page swapping and read data from disk, which will dramatically degrade the Neo4j vector index search performance.
8+
Tools like IOTOP can assist in understanding disk I/O usage.
9+
10+
== Optimal Neo4j memory configuration for vector indexes
11+
12+
The memory configuration recommendations are _Heap + Neo4j PageCache + .25(Vector Index Size) + Additional OS Managed Memory_.
13+
14+
=== Considerations and caveats
15+
16+
You can reduce the Neo4j page cache if you plan not to return vectors to the user or calling application, as the vectors are not needed and should not be loaded into memory.
17+
For example, the vector storage in a database with a total size of 459 GB is 402 GB.
18+
By setting the page cache to 100 GB, the important part of the graph is still in memory, and the server requirements are reduced.
19+
The ratio of memory to storage will be high, but Neo4j will still be able to maintain its performance.
20+
A 1:4 ratio should perform well.
21+
22+
You must increase the memory if you plan to return vectors to the user or call the application and use them for a refined search.
23+
If you use the vectors for further searching or refining search results, the page cache memory allocation must also be increased.
24+
25+
=== Example calculations
26+
27+
The following examples show how to calculate the memory requirements when vectors are only used for searching and will not be returned to the user or application.
28+
29+
.Disk storage requirements
30+
[cols="h,1,1"]
31+
|===
32+
| Neo4j DB
33+
| 10M
34+
| ~40GB
35+
36+
| Vector Index (single index)
37+
| (1.1 * (4 * 768 + 8 * 16) * 10M)/1048576000
38+
| 33.5GB
39+
40+
| Total DB Size
41+
|
42+
| 73.5GB
43+
|===
44+
45+
.Memory requirements
46+
[cols="h,1,1"]
47+
|===
48+
| Heap
49+
| 10-20GB
50+
| 20GB
51+
52+
| Page Cache
53+
| DB Size * 1.2
54+
| 50GB
55+
56+
| OS Memory for Index
57+
| .4 of the Vector Index
58+
| 12GB
59+
60+
| Total
61+
|
62+
| 82GB
63+
|===
64+
65+
.Aura vector specified cluster memory configurations
66+
[options="header", cols="1,1,1,1,1"]
67+
|===
68+
| Instance Size
69+
| Disk Storage
70+
| Heap
71+
| Page Cache
72+
| Remaining Memory
73+
74+
| 32GB
75+
| 64GB
76+
| 7.58GB
77+
| 9.01GB
78+
| 15.41GB
79+
80+
| 64GB
81+
| 128GB
82+
| 16.17GB
83+
| 17.56GB
84+
| 30.27GB
85+
86+
| 128GB
87+
| 256GB
88+
| 26.90GB
89+
| 49.94GB
90+
| 51.16GB
91+
92+
| 256GB
93+
| 512GB
94+
| 31GB
95+
| 132.34GB
96+
| 92.66GB
97+
98+
| 384GB
99+
| 768GB
100+
| 31GB
101+
| 220.25GB
102+
| 132.75GB
103+
104+
| 512GB
105+
| 1024GB
106+
| 31GB
107+
| 308.55GB
108+
| 172.45GB
109+
|===
110+
111+
.Aura non-vector specified cluster memory configurations
112+
[options="header", cols="1,1,1,1,1"]
113+
|===
114+
| Instance Size
115+
| Disk Storage
116+
| Heap
117+
| Page Cache
118+
| Remaining Memory
119+
120+
| 32GB
121+
| 64GB
122+
| 10.39GB
123+
| 11.13GB
124+
| 10.48GB
125+
126+
| 64GB
127+
| 128GB
128+
| 20.57GB
129+
| 23.43GB
130+
| 20GB
131+
132+
| 128GB
133+
| 256GB
134+
| 29.60GB
135+
| 70.40GB
136+
| 28GB
137+
138+
| 256GB
139+
| 512GB
140+
| 31GB
141+
| 180.20GB
142+
| 44.8GB
143+
144+
| 384GB
145+
| 768GB
146+
| 31GB
147+
| 293.20GB
148+
| 59.8GB
149+
150+
| 512GB
151+
| 1024GB
152+
| 31GB
153+
| 410.5GB
154+
| 70.5GB
155+
|===
156+
157+
== Warming up the vector index
158+
159+
The Neo4j vector index is only loaded into memory when it is accessed.
160+
Ideally, the Lucene vector index is preloaded into OS-managed memory before quering the index.
161+
However, you can also warm up the index by running a few random queries to help the OS load the index into memory.
162+
The number of queries required to warm up the index depends on the size of the index and the amount of memory available.
163+
For a smaller index (up to 1M entries), five queries should be sufficient to load the index into memory.

0 commit comments

Comments
 (0)