Elixir crash quitely when loading ONNX model #42
Description
Background and Description
I wanna build elixir's DiffSinger wrapper with ONNX via Ortex. DiffSinger's ONNX version is supported and can be executed on OpenUTAU currently.
But elixir is crashed whitout crash dump or message when I try to load acoustic model(size: 312 MB) that extraced from OptnUTAU's singer lib. The following is a reproduction of that issue.
Livebook Session
# Run as: iex --dot-iex path/to/notebook.exs
# Title: DiffSinger in Elixir via Ortex
Mix.install([{:ortex, "~> 0.1.10"}, {:nx, "~> 0.8"}])
# ── Model preparation ──
# You should modify path within your model's location.
model_root_path = "<bla bla>/Code/QyEditor/priv/Qixuan_v2.5.0_DiffSinger_OpenUtau"
model_path = fn sub -> Path.join(model_root_path, sub) end
# Variance Model
# pitch_predict_path = model_path.("")
linguisitic_path = model_path.("dsvariance/0816_qixuan_multilingual_multivar.qixuan.linguistic.onnx")
variance_path = model_path.("dsvariance/0816_qixuan_multilingual_multivar.qixuan.variance.onnx")
# Acostic Model
acostic_model_path = model_path.("0816_qixuan_multilingual_acoustic.qixuan.onnx")
# Vocoder
vocoder_path = model_path.("dsvocoder/nsf_hifigan_qixuan_004.onnx")
# ── Identify the inputs and outputs of model ──
# Simplest one is vocoder, it receive mel spectrum and pitch, and puts waveform.
# 装载前，进程的内存占用在 21.8M 左右，代码在 25.6M 左右
vocoder = Ortex.load(vocoder_path)
# #Ortex.Model<
# inputs: [
# {"mel",
# "Tensor {
# ty: Float32,
# dimensions: [ 1, -1, 128 ],
# }",
# [1, -1, 128]},
# {"f0", "Tensor {\n ty: Float32,\n dimensions: [\n 1,\n -1,\n ],\n}",
# [1, -1]}
# ]
# outputs: [
# {"waveform",
# "Tensor {\n ty: Float32,\n dimensions: [\n 1,\n -1,\n ],\n}", [1, -1]}
# ]>
# 内存占用没有显著变化。
acostic_model = Ortex.load(acostic_model_path)
# raise
# Runtime terminated unexpectedly - no connection
# ── Preparing inputs ──
before loading the larger model, I loaded a tiny version(less than 100 MB) and it success.
Enviornment
Currently, I use my laptop to coding.
Dependencies
- System: Windows 11
- Processor: i5-8250U
- RAM: 8GB
- Elixir version: 1.16.0 and 1.17.0 (via Livebook) with OTP/26
- Rust version: 1.18.2
- Build chain: Visual Studio 2022
Expectations
- figure out why it crashed?
- Is it fixable?
- [PURELY PERSONAL ISSUE, for the sake of my desktop only has Intel Arc] How to add another execution provider like OpenVINO or OneDNN from source?
Screenshot
