Skip to content

Commit 0ffb32c

Browse files
krishung5tanmayv25
andauthored
Update the doc to point to the tcmalloc/jemalloc usage instructions (#98)
* Update the doc to point to the tcmalloc/jemalloc usage instructions * Update README.md Co-authored-by: Tanmay Verma <[email protected]> --------- Co-authored-by: Tanmay Verma <[email protected]>
1 parent 01efda4 commit 0ffb32c

File tree

1 file changed

+12
-0
lines changed

1 file changed

+12
-0
lines changed

README.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -328,3 +328,15 @@ parameters: {
328328
}
329329
}
330330
```
331+
332+
333+
## Important Notes
334+
* We have observed memory growth issues with the SavedModel format during model
335+
loading and unloading. It is possible that this is not an actual memory leak but
336+
rather a result of the system's malloc heuristics, causing the memory to not be
337+
immediately released back to the operating system. We have noticed improved
338+
memory footprint by replacing the default malloc library with either
339+
[tcmalloc](https://github.com/google/tcmalloc) or
340+
[jemalloc](https://github.com/jemalloc/jemalloc). Please refer to the
341+
[documentation](https://github.com/triton-inference-server/server/blob/main/docs/user_guide/model_management.md#model-control-mode-explicit)
342+
for instructions on how to use tcmalloc or jemalloc with Triton.

0 commit comments

Comments
 (0)