You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Triton exposes some flags to control the execution mode of the TorchScript models through the `Parameters` section of the model's `config.pbtxt` file.
@@ -253,7 +285,6 @@ Triton exposes some flags to control the execution mode of the TorchScript model
253
285
***Additional Optimizations**:
254
286
255
287
Three additional boolean parameters are available to disable certain Torch optimizations that can sometimes cause latency regressions in models with complex execution modes and dynamic shapes.
256
-
257
288
If not specified, all are enabled by default.
258
289
259
290
`ENABLE_JIT_EXECUTOR`
@@ -288,40 +319,6 @@ where the input tensors are placed as follows:
288
319
> [!IMPORTANT]
289
320
> If a device is not specified in the model, the backend uses the first available GPU device.
290
321
291
-
### PyTorch 2.0 Models
292
-
293
-
The model repository should look like:
294
-
295
-
```bash
296
-
model_repository/
297
-
`-- model_directory
298
-
|-- 1
299
-
||-- model.py
300
-
|`-- [model.pt]
301
-
`-- config.pbtxt
302
-
```
303
-
304
-
The `model.py` contains the class definition of the PyTorch model.
0 commit comments