11# Ortex
22
3- ` Ortex ` is a wrapper around [ ONNX Runtime] ( https://onnxruntime.ai/ ) implemented as a
4- (limited) ` Nx.Backend ` using ` Rustler ` and [ ` ort ` ] ( https://github.com/pykeio/ort ) .
3+ ` Ortex ` is a wrapper around [ ONNX Runtime] ( https://onnxruntime.ai/ ) (implemented as
4+ bindings to [ ` ort ` ] ( https://github.com/pykeio/ort ) ). Ortex leverages
5+ [ ` Nx.Serving ` ] ( https://hexdocs.pm/nx/Nx.Serving.html ) to easily deploy ONNX models
6+ that run concurrently and distributed in a cluster. Ortex also provides a storage-only
7+ tensor implementation for ease of use.
58
69ONNX models are a standard machine learning model format that can be exported from most ML
710libraries like PyTorch and TensorFlow. Ortex allows for easy loading and fast inference of
@@ -10,33 +13,36 @@ ML, and ARM Compute Library.
1013
1114## Examples
1215
13- TL;DR
16+ TL;DR:
17+
1418``` elixir
1519iex> model = Ortex .load (" ./models/resnet50.onnx" )
1620# Ortex.Model<
1721 inputs: [{" input" , " Float32" , [nil , 3 , 224 , 224 ]}]
1822 outputs: [{" output" , " Float32" , [nil , 1000 ]}]>
1923iex> {output} = Ortex .run (model, Nx .broadcast (0.0 , {1 , 3 , 224 , 224 }))
20- iex> output |> Nx .backend_transfer (Nx . BinaryBackend ) |> Nx .argmax
24+ iex> output |> Nx .backend_transfer () |> Nx .argmax
2125# Nx.Tensor<
2226 s64
2327 499
2428>
2529```
30+
2631Inspecting a model shows the expected inputs, outputs, data types, and shapes. Axes with
2732` nil ` represent a dynamic size.
2833
29- To see more real world examples see ` examples ` .
34+ To see more real world examples see the ` examples ` folder .
3035
3136### Serving
37+
3238` Ortex ` also implements ` Nx.Serving ` behaviour. To use it in your application's
3339supervision tree consult the ` Nx.Serving ` docs.
3440
3541``` elixir
3642iex> serving = Nx .Serving .new (Ortex .Serving , model)
3743iex> batch = Nx .Batch .stack ([{Nx .broadcast (0.0 , {3 , 224 , 224 })}])
3844iex> {result} = Nx .Serving .run (serving, batch)
39- iex> result |> Nx .backend_transfer |> Nx .argmax (axis: 1 )
45+ iex> result |> Nx .backend_transfer () |> Nx .argmax (axis: 1 )
4046# Nx.Tensor<
4147 s64[1 ]
4248 [499 ]
@@ -54,3 +60,5 @@ def deps do
5460 ]
5561end
5662```
63+
64+ You will need [ Rust] ( https://www.rust-lang.org/tools/install ) for compilation to succeed.
0 commit comments