Skip to content

Commit d708869

Browse files
authored
Exception if remote server is not in explicit mode (#754)
* Throw exception if remote server is not in explicit mode * Fix based on review * Remove comment
1 parent 27da17f commit d708869

File tree

3 files changed

+12
-10
lines changed

3 files changed

+12
-10
lines changed

docs/config_search.md

Lines changed: 0 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -148,14 +148,6 @@ _This will perform an Automatic Brute Search with instance group counts: 3-5, ba
148148

149149
---
150150

151-
### **Interaction with Remote Triton Launch Mode**
152-
153-
When the triton launch mode is remote, _\*\*only concurrency or request rate values can be swept._\*\*<br>
154-
155-
Model Analyzer will ignore any model config parameters because we have no way of accessing and modifying the model repository of the remote Triton Server.
156-
157-
---
158-
159151
## Manual Brute Search
160152

161153
**Default brute search mode when any model config parameters or parameters are specified**

model_analyzer/triton/client/client.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,10 @@ def load_model(self, model_name, variant_name="", config_str=None):
105105
return None
106106
except Exception as e:
107107
logger.info(f"Model {variant_name} load failed: {e}")
108+
if "polling is enabled" in e.message():
109+
raise TritonModelAnalyzerException(
110+
"The remote Tritonserver needs to be launched in EXPLICIT mode"
111+
)
108112
return -1
109113

110114
def unload_model(self, model_name):

tests/mocks/mock_client.py

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,8 @@
1616

1717
from unittest.mock import ANY, MagicMock, Mock, patch
1818

19+
from tritonclient.http import InferenceServerException
20+
1921
from .mock_base import MockBase
2022

2123

@@ -140,8 +142,12 @@ def raise_exception_on_load(self):
140142
InferenceServerException
141143
"""
142144

143-
self.grpc_mock.return_value.load_model.side_effect = Exception
144-
self.http_mock.return_value.load_model.side_effect = Exception
145+
self.grpc_mock.return_value.load_model.side_effect = InferenceServerException(
146+
""
147+
)
148+
self.http_mock.return_value.load_model.side_effect = InferenceServerException(
149+
""
150+
)
145151

146152
def raise_exception_on_unload(self):
147153
"""

0 commit comments

Comments
 (0)