Getting gibberish answers using davinci with Completion endpoint

[mattd1980]

hello,

I was wondering if I was the only one experiencing this, has nothing to do with this library per say but, I'm taking a chance in case someone experienced this :

Here's my code (nothing fancy)

if (input.ToLower() != "exit")
{
var completionResult = await sdk.Completions.CreateCompletion(new CompletionCreateRequest
{
Prompt = input,
Model = Models.Davinci,
});

                if (completionResult.Successful)
                {
                    string response = completionResult.Choices[0].Text;
                    Console.WriteLine("Assistant: " + response);
                }
                else
                {
                    Console.WriteLine("Error: " + completionResult.Error.Message);
                }

Here is the behavior I'm experimenting while using my app:

You: test
Assistant: FreqTable": { "type": "list_of_integers",
You: hello?
Assistant: sp= visiting.20room7.20:.556?cb=t

I even seem to be getting questions from other users as answers, occasionally any have any idea what going on?

[Frogley]

It seems like you're working on a chat app. A more appropriate choice would be to use ChatCompletion instead of Completions. Please refer to this code snippet:

var completionResult = await openAiService.ChatCompletion.CreateCompletion(new ChatCompletionCreateRequest
{
    Messages = new List<ChatMessage>
    {
        ChatMessage.FromSystem("You are a helpful assistant."),
        ChatMessage.FromUser("Who won the world series in 2020?"),
        ChatMessage.FromAssistant("The Los Angeles Dodgers won the World Series in 2020."),
        ChatMessage.FromUser("Where was it played?")
    },
    Model = Models.ChatGpt3_5Turbo,
    MaxTokens = 50//optional
});
if (completionResult.Successful)
{
   Console.WriteLine(completionResult.Choices.First().Message.Content);
}

[mattd1980]

Thank you for your answer but I'm trying to work with a fine tuned model. If I understand correctly, you cannot use chat endpoint, but rather completion. Maybe I'm missing something, that said in my sample I use DaVinci but I get the same behaviour with my custom model.

[Frogley]

Now I get it. For fine-tuning, we currently only have the option of Davinci with Completion mode. You can try organizing the prompt as a conversation like this:

        var completionResult = await openAiService.Completions.CreateCompletion(new CompletionCreateRequest()
        {
            Prompt =
                "The following is a conversation with an AI assistant. \r\n" +
                "The assistant is helpful, creative, clever, and very friendly.\r\n" +
                "Human: What's the most popular sports in the world?\r\n"+
                "AI:",
            Model = Models.Davinci,
            MaxTokens = 150,
            Temperature = 0.9f,
            TopP = 1,
            FrequencyPenalty = 0,
            PresencePenalty = 0.6f,
            StopAsList = new List<string>() { "Human:", "AI:" }
        });
        if (completionResult.Successful)
        {
            Console.WriteLine(completionResult.Choices.FirstOrDefault());
        }

Please note the StopAsList parameter. The output looks like this:

ChoiceResponse { Text = That's easy. Football is the number one sport in the world. , Index = 0, FinishReason = , LogProbs = }

Afterward, you can put this response and the user's next input into the prompt, to enable a continuous conversation.