How do i set temperature when using AskAsync?

[Christofferjensen]

Context / Scenario

How do i set temperature when using AskAsync?

Question

How do i set temperature when using AskAsync?

        var openAIConfig = new OpenAIConfig();
        new ConfigurationBuilder()
            .AddJsonFile("appsettings.json")
            .Build()
            .BindSection("KernelMemory:Services:OpenAI", openAIConfig);

        var memory = new KernelMemoryBuilder()
            .WithOpenAI(openAIConfig)
            .WithSearchClientConfig(new SearchClientConfig {
                AnswerTokens = 400,
                MaxMatchesCount = 10,
            })
            .WithOpenAITextGeneration(openAIConfig)
            .Build<MemoryServerless>();

        Answer = memory.AskAsync(Promt, filters: keyValuePairsColl);

Any help would be appreciated

[dluc]

TopP and Temperature are hard coded in SearchClient. The values are set to 0 to reduce the chance of hallucinations and maximize reproducible answers. If you must change those values, there's a few options and ideas:

1. you can use KM builder's WithCustomSearchClient method to inject a customized version of SearchClient. So you could duplicate that class, maintain your own copy, with your custom settings
2. we would welcome a PR that extends the values in SearchClientConfig to include temperature and other settings.
3. there's a long term plan, no timeline yet, to allow passing in these values during the request. That would/will allow more use cases without the need to change configurations, redeploying the service etc. The approach needs to be designed, so this won't be available soon, so I would considering (1) or (2).

[dluc]

Update: @marcominerva added new settings to SeachClientConfig, see #341, allowing to configure Temperature, TopP and other LLM request settings. These will soon be used by AskAsync, PR coming soon.

[dluc]

Fixed: see #341 and #344

Settings can be configured in appsettings.<env>.json, see appsettings.json for a reference https://github.com/microsoft/kernel-memory/blob/main/service/Service/appsettings.json