Add missing logit_bias field in _rest_call method of OpenAI class in …

[cwang1018]

In the following example, I need to select an element from ['sure', 'likely', 'impossible'].

verifier_program = guidance('''... {{#assistant}}{{select "output" options=valid_output}} {{/assistant}}''', silent=True)

Before the fix:

In the '_rest_call' method, the request body is as follows:
JsonData:

{'model': 'gpt-4-0613', 'max_tokens': 1, 'temperature': 0.0, 'top_p': 1.0, 'n': 1, 'stream': False, 'stop': None, 'messages': [{'role': 'system', 'content': 'Suppose you are one of the greatest AI scientists, logicians and mathematicians. You are very good at basic arithmetic operations. Use numbers and basic arithmetic operations (+ - * /) to obtain 24 with input numbers. Evaluate if given numbers can reach 24 (sure/likely/impossible) '}, {'role': 'user', 'content': '    \nInput: 10, 14\nDraft:\n'}, {'role': 'assistant', 'content': '\n14 - 10 = 4\n14 * 10 = 140\n10 / 14 = 5/7\n14 / 10 = 1.4\n10 + 14 = 24\n'}, {'role': 'user', 'content': '\nOutput:\n'}, {'role': 'assistant', 'content': '\nsure\n10 + 14 = 24\n'}, {'role': 'user', 'content': '    \nInput: 4, 4, 10\nDraft:\n'}, {'role': 'assistant', 'content': '\n10 - 4 + 4 = 6 + 4 = 10\n4 + 4 + 10 = 8 + 10 = 18\n4 * 4 + 10 = 16 + 10 = 26\n4 * 10 - 4 = 40 - 4 = 36\n(10 - 4) * 4 = 6 * 4 = 24\n'}, {'role': 'user', 'content': '\nOutput:\n'}, {'role': 'assistant', 'content': '\nsure\n(10 - 4) * 4 = 6 * 4 = 24\n'}, {'role': 'user', 'content': '    \nInput: 5 15\nDraft:\n'}, {'role': 'assistant', 'content': '\n5 + 15 = 20\n5 - 15 = -10\n5 * 15 = 75\n5 / 15 = 1/3\n15 - 5 = 10\n15 / 5 = 3\nThe input only has two numbers, so I tried all possibility, but no one reached 24.\n'}, {'role': 'user', 'content': '\nOutput:\n'}, {'role': 'assistant', 'content': '\nimpossible\n'}, {'role': 'user', 'content': '    \nInput: 4, 9, 11\nDraft:\n'}, {'role': 'assistant', 'content': '\n4 + 9 + 11 = 24\n'}, {'role': 'user', 'content': '\nOutput:\n'}, {'role': 'assistant', 'content': '\nsure\n4 + 9 + 11 = 24\n'}, {'role': 'user', 'content': '    \nInput: 1, 5, 20\nDraft:\n'}, {'role': 'assistant', 'content': '\n1 + 5 + 20 = 26\n1 * 5 + 20 = 25\n20 / 5 + 1 = 5\n1 * (20 - 5) = 15\n1 * (20 + 5) = 25\n1 - 5 + 20 = 16\n5 - 1 + 20 = 24\n'}, {'role': 'user', 'content': '\nOutput:\n'}, {'role': 'assistant', 'content': '\nsure\n5 - 1 + 20 = 24\n'}, {'role': 'user', 'content': '    \nInput: 1, 2, 11\nDraft:\n'}, {'role': 'assistant', 'content': '\n1 * 2 + 11 = 13\n1 + 2 + 11 = 14\n2 * (11 - 1) = 10\n11 * 2 - 1 = 21\n2 / 1 * 11 = 22\n(11 - 1) / 2 = 5\n11 - 1 * 2 = 9\n1 * (2 + 11) = 13\n1 * (11 - 2) = 9\n2 * 11 + 1 = 22\n2 * (11 + 1) = 24\n'}, {'role': 'user', 'content': '\nOutput:\n'}, {'role': 'assistant', 'content': '\nsure\n2 * (11 + 1) = 24\n'}, {'role': 'user', 'content': '    \nInput: 5, 7, 8\nDraft:\n'}, {'role': 'assistant', 'content': '\n5 * 8 - 7 = 33\n5 * 7 - 8 = 27\n5 * 7 + 8 = 43\n5 * (7 + 8) = 75\n5 + 7 + 8 = 12 + 8 = 20\n(8 - 5) * 7 = 3 * 7 = 21\nI cannot obtain 24 now, but numbers are within a reasonable range\n'}, {'role': 'user', 'content': '\nOutput:\n'}, {'role': 'assistant', 'content': '\nlikely\n'}, {'role': 'user', 'content': '    \nInput: 10, 10, 10\nDraft:\n'}, {'role': 'assistant', 'content': '\n10 10 10 are all too big\n'}, {'role': 'user', 'content': '\nOutput:\n'}, {'role': 'assistant', 'content': '\nimpossible\n'}, {'role': 'user', 'content': '    \nInput: 10, 10, 11\nDraft:\n'}, {'role': 'assistant', 'content': '\n10 + 10 - 11 = 9\n10 * 11 + 10 = 120\n11 * (10 / 10) = 11\n(11 - 10) + 10 = 11\n10 + 10 + 11 = 31\n(11 - 10) * 10 = 10\nI cannot obtain 24 now, but numbers are within a reasonable range\n'}, {'role': 'user', 'content': '\nOutput:\n'}]}

It can be observed that the logit_bias field is missing, and the returned text is '\n'.
Output:

{'finish_reason': 'length', 'index': 0, 'message': {'role': 'assistant', 'content': '\n'}, 'text': '\n'}

After the fix:

JsonData:

{'model': 'gpt-4-0613', 'max_tokens': 1, 'temperature': 0.0, 'top_p': 1.0, 'n': 1, 'stream': False, 'stop': None, 'logit_bias': {'19643': 100, '14617': 100, '318': 100}, 'messages': [{'role': 'system', 'content': 'Suppose you are one of the greatest AI scientists, logicians and mathematicians. You are very good at basic arithmetic operations. Use numbers and basic arithmetic operations (+ - * /) to obtain 24 with input numbers. Evaluate if given numbers can reach 24 (sure/likely/impossible) '}, {'role': 'user', 'content': '    \nInput: 10, 14\nDraft:\n'}, {'role': 'assistant', 'content': '\n14 - 10 = 4\n14 * 10 = 140\n10 / 14 = 5/7\n14 / 10 = 1.4\n10 + 14 = 24\n'}, {'role': 'user', 'content': '\nOutput:\n'}, {'role': 'assistant', 'content': '\nsure\n10 + 14 = 24\n'}, {'role': 'user', 'content': '    \nInput: 4, 4, 10\nDraft:\n'}, {'role': 'assistant', 'content': '\n10 - 4 + 4 = 6 + 4 = 10\n4 + 4 + 10 = 8 + 10 = 18\n4 * 4 + 10 = 16 + 10 = 26\n4 * 10 - 4 = 40 - 4 = 36\n(10 - 4) * 4 = 6 * 4 = 24\n'}, {'role': 'user', 'content': '\nOutput:\n'}, {'role': 'assistant', 'content': '\nsure\n(10 - 4) * 4 = 6 * 4 = 24\n'}, {'role': 'user', 'content': '    \nInput: 5 15\nDraft:\n'}, {'role': 'assistant', 'content': '\n5 + 15 = 20\n5 - 15 = -10\n5 * 15 = 75\n5 / 15 = 1/3\n15 - 5 = 10\n15 / 5 = 3\nThe input only has two numbers, so I tried all possibility, but no one reached 24.\n'}, {'role': 'user', 'content': '\nOutput:\n'}, {'role': 'assistant', 'content': '\nimpossible\n'}, {'role': 'user', 'content': '    \nInput: 4, 9, 11\nDraft:\n'}, {'role': 'assistant', 'content': '\n4 + 9 + 11 = 24\n'}, {'role': 'user', 'content': '\nOutput:\n'}, {'role': 'assistant', 'content': '\nsure\n4 + 9 + 11 = 24\n'}, {'role': 'user', 'content': '    \nInput: 1, 5, 20\nDraft:\n'}, {'role': 'assistant', 'content': '\n1 + 5 + 20 = 26\n1 * 5 + 20 = 25\n20 / 5 + 1 = 5\n1 * (20 - 5) = 15\n1 * (20 + 5) = 25\n1 - 5 + 20 = 16\n5 - 1 + 20 = 24\n'}, {'role': 'user', 'content': '\nOutput:\n'}, {'role': 'assistant', 'content': '\nsure\n5 - 1 + 20 = 24\n'}, {'role': 'user', 'content': '    \nInput: 1, 2, 11\nDraft:\n'}, {'role': 'assistant', 'content': '\n1 * 2 + 11 = 13\n1 + 2 + 11 = 14\n2 * (11 - 1) = 10\n11 * 2 - 1 = 21\n2 / 1 * 11 = 22\n(11 - 1) / 2 = 5\n11 - 1 * 2 = 9\n1 * (2 + 11) = 13\n1 * (11 - 2) = 9\n2 * 11 + 1 = 22\n2 * (11 + 1) = 24\n'}, {'role': 'user', 'content': '\nOutput:\n'}, {'role': 'assistant', 'content': '\nsure\n2 * (11 + 1) = 24\n'}, {'role': 'user', 'content': '    \nInput: 5, 7, 8\nDraft:\n'}, {'role': 'assistant', 'content': '\n5 * 8 - 7 = 33\n5 * 7 - 8 = 27\n5 * 7 + 8 = 43\n5 * (7 + 8) = 75\n5 + 7 + 8 = 12 + 8 = 20\n(8 - 5) * 7 = 3 * 7 = 21\nI cannot obtain 24 now, but numbers are within a reasonable range\n'}, {'role': 'user', 'content': '\nOutput:\n'}, {'role': 'assistant', 'content': '\nlikely\n'}, {'role': 'user', 'content': '    \nInput: 10, 10, 10\nDraft:\n'}, {'role': 'assistant', 'content': '\n10 10 10 are all too big\n'}, {'role': 'user', 'content': '\nOutput:\n'}, {'role': 'assistant', 'content': '\nimpossible\n'}, {'role': 'user', 'content': '    \nInput: 10, 10, 11\nDraft:\n'}, {'role': 'assistant', 'content': '\n10 + 10 - 11 = 9\n10 * 11 + 10 = 120\n11 * (10 / 10) = 11\n(11 - 10) + 10 = 11\n10 + 10 + 11 = 31\n(11 - 10) * 10 = 10\nI cannot obtain 24 now, but numbers are within a reasonable range\n'}, {'role': 'user', 'content': '\nOutput:\n'}]}

The request body now includes the logit_bias field, and the logit_bias is defined as {'19643': 100, '14617': 100, '318': 100} during runtime. The output text is 'likely', which is as expected.
Output:

{'finish_reason': 'length', 'index': 0, 'message': {'role': 'assistant', 'content': 'likely'}, 'text': 'likely'}