RuntimeError: Infer Request is busy

[blizaga]

Feb 20 13:07:48 barium sentiment-extractor-server-statement-v2[3228361]: File "/home/bino/app/sentiment-extractor/model_engine/base.py", line 44, in infer
Feb 20 13:07:48 barium sentiment-extractor-server-statement-v2[3228361]: result = self._infer_request.infer(inputs=inputs)
Feb 20 13:07:48 barium sentiment-extractor-server-statement-v2[3228361]: File "/home/bino/.virtualenvs/py3.10_sentiment-v2/lib/python3.10/site-packages/openvino/runtime/ie_api.py", line 144, in infer
Feb 20 13:07:48 barium sentiment-extractor-server-statement-v2[3228361]: return OVDict(super().infer(_data_dispatch(
Feb 20 13:07:48 barium sentiment-extractor-server-statement-v2[3228361]: File "/home/bino/.virtualenvs/py3.10_sentiment-v2/lib/python3.10/site-packages/openvino/runtime/utils/data_helpers/data_dispatcher.py", line 426, in _data_dispatch
Feb 20 13:07:48 barium sentiment-extractor-server-statement-v2[3228361]: return create_shared(inputs, request) if is_shared else create_copied(inputs, request)
Feb 20 13:07:48 barium sentiment-extractor-server-statement-v2[3228361]: File "/usr/lib/python3.10/functools.py", line 889, in wrapper
Feb 20 13:07:48 barium sentiment-extractor-server-statement-v2[3228361]: return dispatch(args[0].class)(*args, **kw)
Feb 20 13:07:48 barium sentiment-extractor-server-statement-v2[3228361]: File "/home/bino/.virtualenvs/py3.10_sentiment-v2/lib/python3.10/site-packages/openvino/runtime/utils/data_helpers/data_dispatcher.py", line 380, in _
Feb 20 13:07:48 barium sentiment-extractor-server-statement-v2[3228361]: return update_inputs(normalize_arrays(inputs, is_shared=False), request)
Feb 20 13:07:48 barium sentiment-extractor-server-statement-v2[3228361]: File "/home/bino/.virtualenvs/py3.10_sentiment-v2/lib/python3.10/site-packages/openvino/runtime/utils/data_helpers/data_dispatcher.py", line 348, in update_inputs
Feb 20 13:07:48 barium sentiment-extractor-server-statement-v2[3228361]: update_tensor(value, request, key)
Feb 20 13:07:48 barium sentiment-extractor-server-statement-v2[3228361]: File "/usr/lib/python3.10/functools.py", line 889, in wrapper
Feb 20 13:07:48 barium sentiment-extractor-server-statement-v2[3228361]: return dispatch(args[0].class)(*args, **kw)
Feb 20 13:07:48 barium sentiment-extractor-server-statement-v2[3228361]: File "/home/bino/.virtualenvs/py3.10_sentiment-v2/lib/python3.10/site-packages/openvino/runtime/utils/data_helpers/data_dispatcher.py", line 297, in _
Feb 20 13:07:48 barium sentiment-extractor-server-statement-v2[3228361]: tensor = get_request_tensor(request, key)
Feb 20 13:07:48 barium sentiment-extractor-server-statement-v2[3228361]: File "/home/bino/.virtualenvs/py3.10_sentiment-v2/lib/python3.10/site-packages/openvino/runtime/utils/data_helpers/data_dispatcher.py", line 39, in get_request_tensor
Feb 20 13:07:48 barium sentiment-extractor-server-statement-v2[3228361]: return request.get_tensor(key)

Feb 20 13:07:48 barium sentiment-extractor-server-statement-v2[3228361]: RuntimeError: Infer Request is busy

this inference busy but why, how to solve it

[ilya-lavrenov]

Looks like the same request is called for inference multiple times
E.g. start_async is called second time, while the request is still processing the previous frame.

[blizaga]

def infer(input_text):
    """
    Creating a generic inference function
    to read the input and infer the result
    into 2 classes: Positive or Negative.
    Parameters:
        input_text (str): The text to be processed.
        
    Returns:
        str: The label of the sentiment analysis result, which can be "Positive", "Negative", or "Neutral".
   """
    input_text = tokenizer(
        input_text,
        truncation=True,
        max_length=512,
        return_tensors="np",
    )
    inputs = dict(input_text)
    label = {0: "positive", 1: "neutral", 2: "negative"}
    result = infer_request.infer(inputs=inputs)
    for i in result.values():
        probability = np.argmax(softmax(i))
    return label[probability]

This my function for inference