Item data size too large for memory

[shriramters]

I was trying to run the decoder on some of the firefox-translations models

Here, I'm running the ende model

Is there any way I can fix the error?

[2025-02-08 07:53:05] [marian] Marian v1.12.0 65bf82ff 2023-02-21 09:56:29 -0800
[2025-02-08 07:53:05] [marian] Running on think182 as process 7910 with command line:
[2025-02-08 07:53:05] [marian] ./marian-decoder -m model.ende.intgemm.alphas.bin -v vocab.deen.spm vocab.deen.spm --shortlist lex.50.50.ende.s2t.bin 50 50
[2025-02-08 07:53:05] [config] alignment: ""
[2025-02-08 07:53:05] [config] allow-special: false
[2025-02-08 07:53:05] [config] allow-unk: false
[2025-02-08 07:53:05] [config] authors: false
[2025-02-08 07:53:05] [config] beam-size: 12
[2025-02-08 07:53:05] [config] bert-class-symbol: "[CLS]"
[2025-02-08 07:53:05] [config] bert-mask-symbol: "[MASK]"
[2025-02-08 07:53:05] [config] bert-masking-fraction: 0.15
[2025-02-08 07:53:05] [config] bert-sep-symbol: "[SEP]"
[2025-02-08 07:53:05] [config] bert-train-type-embeddings: true
[2025-02-08 07:53:05] [config] bert-type-vocab-size: 2
[2025-02-08 07:53:05] [config] best-deep: false
[2025-02-08 07:53:05] [config] build-info: ""
[2025-02-08 07:53:05] [config] check-nan: false
[2025-02-08 07:53:05] [config] cite: false
[2025-02-08 07:53:05] [config] cpu-threads: 1
[2025-02-08 07:53:05] [config] data-threads: 8
[2025-02-08 07:53:05] [config] dec-cell: ssru
[2025-02-08 07:53:05] [config] dec-cell-base-depth: 2
[2025-02-08 07:53:05] [config] dec-cell-high-depth: 1
[2025-02-08 07:53:05] [config] dec-depth: 2
[2025-02-08 07:53:05] [config] devices:
[2025-02-08 07:53:05] [config]   - 0
[2025-02-08 07:53:05] [config] dim-emb: 256
[2025-02-08 07:53:05] [config] dim-rnn: 1024
[2025-02-08 07:53:05] [config] dim-vocabs:
[2025-02-08 07:53:05] [config]   - 32000
[2025-02-08 07:53:05] [config]   - 32000
[2025-02-08 07:53:05] [config] dump-config: ""
[2025-02-08 07:53:05] [config] enc-cell: gru
[2025-02-08 07:53:05] [config] enc-cell-depth: 1
[2025-02-08 07:53:05] [config] enc-depth: 6
[2025-02-08 07:53:05] [config] enc-type: bidirectional
[2025-02-08 07:53:05] [config] factors-combine: sum
[2025-02-08 07:53:05] [config] factors-dim-emb: 0
[2025-02-08 07:53:05] [config] force-decode: false
[2025-02-08 07:53:05] [config] gemm-type: float32
[2025-02-08 07:53:05] [config] ignore-model-config: false
[2025-02-08 07:53:05] [config] input:
[2025-02-08 07:53:05] [config]   - stdin
[2025-02-08 07:53:05] [config] input-types:
[2025-02-08 07:53:05] [config]   []
[2025-02-08 07:53:05] [config] interpolate-env-vars: false
[2025-02-08 07:53:05] [config] layer-normalization: false
[2025-02-08 07:53:05] [config] lemma-dependency: ""
[2025-02-08 07:53:05] [config] lemma-dim-emb: 0
[2025-02-08 07:53:05] [config] log: ""
[2025-02-08 07:53:05] [config] log-level: info
[2025-02-08 07:53:05] [config] log-time-zone: ""
[2025-02-08 07:53:05] [config] max-length: 1000
[2025-02-08 07:53:05] [config] max-length-crop: false
[2025-02-08 07:53:05] [config] max-length-factor: 3
[2025-02-08 07:53:05] [config] maxi-batch: 1
[2025-02-08 07:53:05] [config] maxi-batch-sort: none
[2025-02-08 07:53:05] [config] mini-batch: 1
[2025-02-08 07:53:05] [config] mini-batch-words: 0
[2025-02-08 07:53:05] [config] models:
[2025-02-08 07:53:05] [config]   - model.ende.intgemm.alphas.bin
[2025-02-08 07:53:05] [config] n-best: false
[2025-02-08 07:53:05] [config] no-spm-decode: false
[2025-02-08 07:53:05] [config] normalize: 0
[2025-02-08 07:53:05] [config] num-devices: 0
[2025-02-08 07:53:05] [config] optimize: false
[2025-02-08 07:53:05] [config] output: stdout
[2025-02-08 07:53:05] [config] output-approx-knn:
[2025-02-08 07:53:05] [config]   []
[2025-02-08 07:53:05] [config] output-omit-bias: false
[2025-02-08 07:53:05] [config] output-sampling:
[2025-02-08 07:53:05] [config]   []
[2025-02-08 07:53:05] [config] precision:
[2025-02-08 07:53:05] [config]   - float32
[2025-02-08 07:53:05] [config] quantize-range: 0
[2025-02-08 07:53:05] [config] quiet: false
[2025-02-08 07:53:05] [config] quiet-translation: false
[2025-02-08 07:53:05] [config] relative-paths: false
[2025-02-08 07:53:05] [config] right-left: false
[2025-02-08 07:53:05] [config] seed: 0
[2025-02-08 07:53:05] [config] shortlist:
[2025-02-08 07:53:05] [config]   - lex.50.50.ende.s2t.bin
[2025-02-08 07:53:05] [config]   - 50
[2025-02-08 07:53:05] [config]   - 50
[2025-02-08 07:53:05] [config] skip: false
[2025-02-08 07:53:05] [config] skip-cost: false
[2025-02-08 07:53:05] [config] stat-freq: 0
[2025-02-08 07:53:05] [config] tied-embeddings: false
[2025-02-08 07:53:05] [config] tied-embeddings-all: true
[2025-02-08 07:53:05] [config] tied-embeddings-src: false
[2025-02-08 07:53:05] [config] transformer-aan-activation: swish
[2025-02-08 07:53:05] [config] transformer-aan-depth: 2
[2025-02-08 07:53:05] [config] transformer-aan-nogate: false
[2025-02-08 07:53:05] [config] transformer-decoder-autoreg: rnn
[2025-02-08 07:53:05] [config] transformer-decoder-dim-ffn: 0
[2025-02-08 07:53:05] [config] transformer-decoder-ffn-depth: 0
[2025-02-08 07:53:05] [config] transformer-depth-scaling: false
[2025-02-08 07:53:05] [config] transformer-dim-aan: 2048
[2025-02-08 07:53:05] [config] transformer-dim-ffn: 1536
[2025-02-08 07:53:05] [config] transformer-ffn-activation: relu
[2025-02-08 07:53:05] [config] transformer-ffn-depth: 2
[2025-02-08 07:53:05] [config] transformer-guided-alignment-layer: last
[2025-02-08 07:53:05] [config] transformer-heads: 8
[2025-02-08 07:53:05] [config] transformer-no-projection: false
[2025-02-08 07:53:05] [config] transformer-pool: false
[2025-02-08 07:53:05] [config] transformer-postprocess: dan
[2025-02-08 07:53:05] [config] transformer-postprocess-emb: d
[2025-02-08 07:53:05] [config] transformer-postprocess-top: ""
[2025-02-08 07:53:05] [config] transformer-preprocess: ""
[2025-02-08 07:53:05] [config] transformer-rnn-projection: false
[2025-02-08 07:53:05] [config] transformer-tied-layers:
[2025-02-08 07:53:05] [config]   []
[2025-02-08 07:53:05] [config] transformer-train-position-embeddings: false
[2025-02-08 07:53:05] [config] tsv: false
[2025-02-08 07:53:05] [config] tsv-fields: 0
[2025-02-08 07:53:05] [config] type: transformer
[2025-02-08 07:53:05] [config] ulr: false
[2025-02-08 07:53:05] [config] ulr-dim-emb: 0
[2025-02-08 07:53:05] [config] ulr-trainable-transformation: false
[2025-02-08 07:53:05] [config] version: v1.9.56 dad4d50 2021-06-28 12:10:40 +0100
[2025-02-08 07:53:05] [config] vocabs:
[2025-02-08 07:53:05] [config]   - vocab.deen.spm
[2025-02-08 07:53:05] [config]   - vocab.deen.spm
[2025-02-08 07:53:05] [config] weights:
[2025-02-08 07:53:05] [config]   []
[2025-02-08 07:53:05] [config] word-penalty: 0
[2025-02-08 07:53:05] [config] word-scores: false
[2025-02-08 07:53:05] [config] workspace: 512
[2025-02-08 07:53:05] [config] Loaded model has been created with Marian v1.9.56 dad4d50 2021-06-28 12:10:40 +0100
[2025-02-08 07:53:05] [data] Loading SentencePiece vocabulary from file vocab.deen.spm
[2025-02-08 07:53:05] [data] Loading SentencePiece vocabulary from file vocab.deen.spm
[2025-02-08 07:53:05] [data] Loading binary shortlist as lex.50.50.ende.s2t.bin true
[2025-02-08 07:53:05] [data] Lexical short list firstNum 50 and bestNum 50
[2025-02-08 07:53:05] Loading model from model.ende.intgemm.alphas.bin
[2025-02-08 07:53:05] [memory] Extending reserved space to 512 MB (device cpu0)
[2025-02-08 07:53:06] Loaded model config
[2025-02-08 07:53:06] Loading scorer of type transformer as feature F0
[2025-02-08 07:53:06] [memory] Reserving 270592 B, device cpu0
[2025-02-08 07:53:06] [memory] Reserving 16 MB, device cpu0
[2025-02-08 07:53:06] Error: Item data size 8192256 too large for memory 8192000
[2025-02-08 07:53:06] Error: Aborted from void marian::TensorBase::set(const marian::io::Item&) in /home/shriram/Workspace/marian/src/tensors/tensor.cpp:134

Call Stack

[CALL STACK]
[0x55f26dc36099]    marian::TensorBase::  set  (marian::io::Item const&) + 0xa89
[0x55f26daf8dae]                                                       + 0x551dae
[0x55f26db02d0c]    marian::inits::LambdaInitConvert::  apply  (IntrusivePtr<marian::TensorBase>) + 0x6c
[0x55f26daf4e3f]    marian::ParamNode::  init  ()                      + 0x3f
[0x55f26d96c4bf]    marian::ExpressionGraph::  forward  (std::__cxx11::list<IntrusivePtr<marian::Chainable<IntrusivePtr<marian::TensorBase>>>,std::allocator<IntrusivePtr<marian::Chainable<IntrusivePtr<marian::TensorBase>>>>>&,  bool) + 0x4f
[0x55f26d81520d]    marian::Translate<marian::BeamSearch>::Translate(std::shared_ptr<marian::Options>)::{lambda(marian::DeviceId,unsigned long)#1}::  operator()  (marian::DeviceId,  unsigned long) const + 0x6bd
[0x55f26d815d72]    marian::ThreadPool::enqueue<marian::Translate<marian::BeamSearch>::Translate(std::shared_ptr<marian::Options>)::{lambda(marian::DeviceId,unsigned long)#1}&,marian::DeviceId&,unsigned long>(marian::Translate<marian::BeamSearch>::Translate(std::shared_ptr<marian::Options>)::{lambda(marian::DeviceId,unsigned long)#1}&,marian::DeviceId&,unsigned long&&)::{lambda()#1}::  operator()  () const + 0x22
[0x55f26d816a8e]    std::_Function_handler<std::unique_ptr<std::__future_base::_Result_base,std::__future_base::_Result_base::_Deleter> (),std::__future_base::_Task_setter<std::unique_ptr<std::__future_base::_Result<void>,std::__future_base::_Result_base::_Deleter>,std::__future_base::_Task_state<marian::ThreadPool::enqueue<marian::Translate<marian::BeamSearch>::Translate(std::shared_ptr<marian::Options>)::{lambda(marian::DeviceId,unsigned long)#1}&,marian::DeviceId&,unsigned long>(marian::Translate<marian::BeamSearch>::Translate(std::shared_ptr<marian::Options>)::{lambda(marian::DeviceId,unsigned long)#1}&,marian::DeviceId&,unsigned long&&)::{lambda()#1},std::allocator<int>,void ()>::_M_run()::{lambda()#1},void>>::  _M_invoke  (std::_Any_data const&) + 0x1e
[0x55f26d7b049b]    std::__future_base::_State_baseV2::  _M_do_set  (std::function<std::unique_ptr<std::__future_base::_Result_base,std::__future_base::_Result_base::_Deleter> ()>*,  bool*) + 0x1b
[0x7f2ec7dce027]                                                       + 0x8e027
[0x55f26d7b1560]    std::__future_base::_Task_state<marian::ThreadPool::enqueue<marian::Translate<marian::BeamSearch>::Translate(std::shared_ptr<marian::Options>)::{lambda(marian::DeviceId,unsigned long)#1}&,marian::DeviceId&,unsigned long>(marian::Translate<marian::BeamSearch>::Translate(std::shared_ptr<marian::Options>)::{lambda(marian::DeviceId,unsigned long)#1}&,marian::DeviceId&,unsigned long&&)::{lambda()#1},std::allocator<int>,void ()>::  _M_run  () + 0xf0
[0x55f26d7c742b]    std::thread::_State_impl<std::thread::_Invoker<std::tuple<marian::ThreadPool::reserve(unsigned long)::{lambda()#1}>>>::  _M_run  () + 0x17b
[0x7f2ec80d44a3]                                                       + 0xd44a3
[0x7f2ec7dc91c4]                                                       + 0x891c4
[0x7f2ec7e4985c]                                                       + 0x10985c

Aborted

[shriramters]

Okay, I just had to load the model using mmap,
I fixed it by adding --model-mmap

./marian-decoder -m model.ende.intgemm.alphas.bin --model-mmap -v vocab.deen.spm vocab.deen.spm --shortlist lex.50.50.ende.s2t.bin false --cpu-threads 2 < input.txt