Pad the rows when converting dataframes to dictionaries for Triton requests #234
Description
Bug description
I am trying to serve a TF4Rec model to TIS with Torch backend. When I send a request to TIS, I get the following error if my df is not padded.
response = run_ensemble_on_tritonserver('./models', input_schema, df[input_schema.column_names], output_schema.column_names, "ensemble_model")
ValueError Traceback (most recent call last)
Cell In [36], line 4
1 # ===========================================
2 # Send request to Triton and check response
3 # ===========================================
----> 4 response = run_ensemble_on_tritonserver(
5 './models', input_schema, train_df[input_schema.column_names], output_schema.column_names, "ensemble_model"
6 )
File /usr/local/lib/python3.8/dist-packages/merlin/systems/triton/utils.py:139, in run_ensemble_on_tritonserver(tmpdir, schema, df, output_columns, model_name)
137 response = None
138 with run_triton_server(tmpdir) as client:
--> 139 response = send_triton_request(
140 schema, df, output_columns, client=client, triton_model=model_name
141 )
143 return response
File /usr/local/lib/python3.8/dist-packages/merlin/systems/triton/utils.py:188, in send_triton_request(schema, df, outputs_list, client, endpoint, request_id, triton_model)
185 if not client.is_server_live():
186 raise ValueError("Client could not establish commuincation with Triton Inference Server.")
--> 188 inputs = triton.convert_df_to_triton_input(schema, df, grpcclient.InferInput)
190 outputs = [grpcclient.InferRequestedOutput(col) for col in outputs_list]
191 with client:
File /usr/local/lib/python3.8/dist-packages/merlin/systems/triton/__init__.py:58, in convert_df_to_triton_input(schema, batch, input_class, dtype)
38 def convert_df_to_triton_input(schema, batch, input_class=grpcclient.InferInput, dtype="int32"):
39 """
40 Convert a dataframe to a set of Triton inputs
41
(...)
56 A list of Triton inputs of the requested input class
57 """
---> 58 df_dict = _convert_df_to_dict(schema, batch, dtype)
59 inputs = [
60 _convert_column_to_triton_input(col_name, col_values, input_class)
61 for col_name, col_values in df_dict.items()
62 ]
63 return inputs
File /usr/local/lib/python3.8/dist-packages/merlin/systems/triton/__init__.py:84, in _convert_df_to_dict(schema, batch, dtype)
82 else:
83 values = col.list.leaves.values_host
---> 84 values = values.reshape(*shape).astype(col_schema.dtype)
85 df_dict[col_name] = values
87 else:
ValueError: cannot reshape array of size 63 into shape (5,newaxis)
Steps/Code to reproduce bug
Repro code will be provided in a gist.
Expected behavior
convert_df_to_triton_input function should handle the non-padded, variable length sequences, so may be the simplest way is to create a padding script in this function so we dont get this reshape error.
Environment details
- Merlin version:
- Platform:
- Python version:
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
Merlin-pytorch:22.10 with latest main branches pulled.