You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Triton exposes some flags to control the execution mode of the TorchScript models through the `Parameters` section of the model's `config.pbtxt` file.
@@ -248,7 +280,6 @@ Triton exposes some flags to control the execution mode of the TorchScript model
248
280
***Additional Optimizations**:
249
281
250
282
Three additional boolean parameters are available to disable certain Torch optimizations that can sometimes cause latency regressions in models with complex execution modes and dynamic shapes.
251
-
252
283
If not specified, all are enabled by default.
253
284
254
285
`ENABLE_JIT_EXECUTOR`
@@ -283,40 +314,6 @@ where the input tensors are placed as follows:
283
314
> [!IMPORTANT]
284
315
> If a device is not specified in the model, the backend uses the first available GPU device.
285
316
286
-
### PyTorch 2.0 Models
287
-
288
-
The model repository should look like:
289
-
290
-
```bash
291
-
model_repository/
292
-
`-- model_directory
293
-
|-- 1
294
-
||-- model.py
295
-
|`-- [model.pt]
296
-
`-- config.pbtxt
297
-
```
298
-
299
-
The `model.py` contains the class definition of the PyTorch model.
0 commit comments