Skip to content

Commit 5e9c997

Browse files
authored
Updtae README and versions for 24.10 (#938)
1 parent 463df9e commit 5e9c997

File tree

2 files changed

+2
-114
lines changed

2 files changed

+2
-114
lines changed

README.md

Lines changed: 1 addition & 113 deletions
Original file line numberDiff line numberDiff line change
@@ -20,116 +20,4 @@ limitations under the License.
2020

2121
> [!Warning]
2222
>
23-
> ##### LATEST RELEASE
24-
>
25-
> You are currently on the `main` branch which tracks under-development progress towards the next release. <br>
26-
> The latest release of the Triton Model Analyzer is 1.44.0 and is available on branch
27-
> [r24.09](https://github.com/triton-inference-server/model_analyzer/tree/r24.09).
28-
29-
Triton Model Analyzer is a CLI tool which can help you find a more optimal configuration, on a given piece of hardware, for single, multiple, ensemble, or BLS models running on a [Triton Inference Server](https://github.com/triton-inference-server/server/). Model Analyzer will also generate reports to help you better understand the trade-offs of the different configurations along with their compute and memory requirements.
30-
<br><br>
31-
32-
# Features
33-
34-
### Search Modes
35-
36-
- [Optuna Search](docs/config_search.md#optuna-search-mode) **_-ALPHA RELEASE-_** allows you to search for every parameter that can be specified in the model configuration, using a hyperparameter optimization framework. Please see the [Optuna](https://optuna.org/) website if you are interested in specific details on how the algorithm functions.
37-
38-
- [Quick Search](docs/config_search.md#quick-search-mode) will **sparsely** search the [Max Batch Size](https://github.com/triton-inference-server/server/blob/main/docs/user_guide/model_configuration.md#maximum-batch-size),
39-
[Dynamic Batching](https://github.com/triton-inference-server/server/blob/main/docs/user_guide/model_configuration.md#dynamic-batcher), and
40-
[Instance Group](https://github.com/triton-inference-server/server/blob/main/docs/user_guide/model_configuration.md#instance-groups) spaces by utilizing a heuristic hill-climbing algorithm to help you quickly find a more optimal configuration
41-
42-
- [Automatic Brute Search](docs/config_search.md#automatic-brute-search) will **exhaustively** search the
43-
[Max Batch Size](https://github.com/triton-inference-server/server/blob/main/docs/user_guide/model_configuration.md#maximum-batch-size),
44-
[Dynamic Batching](https://github.com/triton-inference-server/server/blob/main/docs/user_guide/model_configuration.md#dynamic-batcher), and
45-
[Instance Group](https://github.com/triton-inference-server/server/blob/main/docs/user_guide/model_configuration.md#instance-groups)
46-
parameters of your model configuration
47-
48-
- [Manual Brute Search](docs/config_search.md#manual-brute-search) allows you to create manual sweeps for every parameter that can be specified in the model configuration
49-
50-
### Model Types
51-
52-
- [Ensemble](docs/model_types.md#ensemble): Model Analyzer can help you find the optimal
53-
settings when profiling an ensemble model
54-
55-
- [BLS](docs/model_types.md#bls): Model Analyzer can help you find the optimal
56-
settings when profiling a BLS model
57-
58-
- [Multi-Model](docs/model_types.md#multi-model): Model Analyzer can help you
59-
find the optimal settings when profiling multiple concurrent models
60-
61-
- [LLM](docs/model_types.md#llm): Model Analyzer can help you
62-
find the optimal settings when profiling Large Language Models
63-
64-
### Other Features
65-
66-
- [Detailed and summary reports](docs/report.md): Model Analyzer is able to generate
67-
summarized and detailed reports that can help you better understand the trade-offs
68-
between different model configurations that can be used for your model.
69-
70-
- [QoS Constraints](docs/config.md#constraint): Constraints can help you
71-
filter out the Model Analyzer results based on your QoS requirements. For
72-
example, you can specify a latency budget to filter out model configurations
73-
that do not satisfy the specified latency threshold.
74-
<br><br>
75-
76-
# Examples and Tutorials
77-
78-
### **Single Model**
79-
80-
See the [Single Model Quick Start](docs/quick_start.md) for a guide on how to use Model Analyzer to profile, analyze and report on a simple PyTorch model.
81-
82-
### **Multi Model**
83-
84-
See the [Multi-model Quick Start](docs/mm_quick_start.md) for a guide on how to use Model Analyzer to profile, analyze and report on two models running concurrently on the same GPU.
85-
86-
### **Ensemble Model**
87-
88-
See the [Ensemble Model Quick Start](docs/ensemble_quick_start.md) for a guide on how to use Model Analyzer to profile, analyze and report on a simple Ensemble model.
89-
90-
### **BLS Model**
91-
92-
See the [BLS Model Quick Start](docs/bls_quick_start.md) for a guide on how to use Model Analyzer to profile, analyze and report on a simple BLS model.
93-
<br><br>
94-
95-
# Documentation
96-
97-
- [Installation](docs/install.md)
98-
- [Model Analyzer CLI](docs/cli.md)
99-
- [Launch Modes](docs/launch_modes.md)
100-
- [Configuring Model Analyzer](docs/config.md)
101-
- [Model Analyzer Metrics](docs/metrics.md)
102-
- [Model Config Search](docs/config_search.md)
103-
- [Model Types](docs/model_types.md)
104-
- [Checkpointing](docs/checkpoints.md)
105-
- [Model Analyzer Reports](docs/report.md)
106-
- [Deployment with Kubernetes](docs/kubernetes_deploy.md)
107-
<br><br>
108-
109-
# Terminology
110-
111-
Below are definitions of some commonly used terms in Model Analyzer:
112-
113-
- **Model Type** - Category of model being profiled. Examples of this include single, multi, ensemble, BLS, etc..
114-
- **Search Mode** - How Model Analyzer explores the possible configuration space when profiling. This is either exhaustive (brute) or heuristic (quick/optuna).
115-
- **Model Config Search** - The cross product of model type and search mode.
116-
- **Launch Mode** - How the Triton Server is deployed and used by Model Analyzer.
117-
118-
# Reporting problems, asking questions
119-
120-
We appreciate any feedback, questions or bug reporting regarding this
121-
project. When help with code is needed, follow the process outlined in
122-
the Stack Overflow (https://stackoverflow.com/help/mcve)
123-
document. Ensure posted examples are:
124-
125-
- minimal – use as little code as possible that still produces the
126-
same problem
127-
128-
- complete – provide all parts needed to reproduce the problem. Check
129-
if you can strip external dependency and still show the problem. The
130-
less time we spend on reproducing problems the more time we have to
131-
fix it
132-
133-
- verifiable – test the code you're about to provide to make sure it
134-
reproduces the problem. Remove all other problems that are not
135-
related to your request/question.
23+
> You are currently on the `r24.10` branch which tracks under-development progress towards the next release. <br>

VERSION

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
1.45.0dev
1+
1.45.0

0 commit comments

Comments
 (0)