Using tools / functions with this helper

[gaborvar]

Works great when the model communicates only with the well known "assistant" and "user" messages. However, when "tool"-related messages are also sent, two parameters are omitted which causes an error a few lines later.

openai-messages-token-helper/src/openai_messages_token_helper/message_builder.py

Line 152 in eb59b9a

message_builder.insert_message(message["role"], message["content"], index=append_index) # type: ignore[arg-type]

The code handles the same task correctly when passing few-shot examples. It lends an easy fix for workload messages. The same two parameters tool_calls and tool_call_id are picked up there as expected by other parts of the flow.
I use the OpenAI sample app from https://github.com/Azure-Samples/azure-search-openai-demo which relies on this lib.

[gaborvar]

If tool call request messages are handled this error is hit:

openai-messages-token-helper/src/openai_messages_token_helper/model_helper.py

Line 122 in eb59b9a

raise ValueError(f"Could not encode unsupported message value type: {type(value)}")

Some condition checking code would help:

        elif (key in ["content", "refusal", "function_call"] and value is None 
            and message.get("role")=="assistant" 
            and message.get("tool_calls") is not None):     # If the model calls a tool/function then content can be None
            num_tokens += 0     #   a plausible guess, to be confirmed

[pamelafox]

Do you happen to have the message history you're using? That way I can write a regression test easily and fix this up. Otherwise I'll make one up myself.

[gaborvar]

I uploaded to this repo two python files with my fixes that address the above mentioned and some other tool call related issues :
https://github.com/gaborvar/openai-messages-token-helper/blob/main/src/openai_messages_token_helper/model_helper.py
https://github.com/gaborvar/openai-messages-token-helper/blob/main/src/openai_messages_token_helper/message_builder.py
Do you prefer a pull req?
Note: My goal was limited to making the helper transparent to tool (function) calls. Impact of tool calls e.g. function argument list on token count has to be accounted for in future refinement. I am not familiar with token counting.

I can also share a message history but I don't have it at hand - need some time to start it up.

[gaborvar]

Hi @pamelafox, Here is a typical message history. Note the pair of messages on the 4th and 5th place which does not get through the helper:

{'content': 'pers num: 1 5610054514', 'role': 'user'}, {'content': "Thank you for providing the details. Here's a summary of the information:\n\n- First Name: John\n- Last Name: Doe\n- Street Address: 231 Any street\n- House Number: 231\n- City of Residence: Budapest\n- Postcode: 1121\n- Mother's Name: Jane Small\n- City of Birth: Abda\n- Date of Birth: October 5, 1956\n- Personal Number (személyi szám): 1 5610054514\n\nPlease confirm that all the details are correct, and I'll proceed with creating the document.", 'role': 'assistant'}, {'content': 'yes', 'role': 'user'}, {'content': None, 'refusal': None, 'role': 'assistant', 'function_call': None, 'tool_calls': [{'id': 'call_rtO3sdC9sLYzTPPaYAat78IG', 'function': {'arguments': '{"lastname":"Doe","firstname":"John","residence_streetaddress":"231 Any street","residence_housenumber":"231","residence_city":"Budapest","postcode":"1121","mother_name":"Jane Small","city_born":"Abda","date_born":"October 5, 1956","szig_num":"1 5610054514","confirmed_by_user":true}', 'name': 'create_doc_from_template_and_details'}, 'type': 'function'}]}, {'role': 'tool', 'content': '{"create_doc_from_template_and_details": false, "Document_URL": "Doe_ugyv_meghat_preproc19._prsnl_1.docx", "Result": false}', 'tool_call_id': 'call_rtO3sdC9sLYzTPPaYAat78IG'}, {'content': 'thank you', 'role': 'user'}

thank you and let me know if you need more background

[gaborvar]

#21 pull request created.
Message_builder() fix reuses the same approach that enables few-shot examples to include tool calls.
Thank you very much!