How to Set Up C# SemanticKernel Planner with Phi3-128k for Dockerfile Analysis

[weldpua2008]

My goal is to establish a local setup for the C# SemanticKernel Planner with Phi3-128k to facilitate Dockerfile analysis. The plan involves leveraging SemanticKernel plugins to clone a git repository and scrutinize each Dockerfile. In instances where a Dockerfile is devoid of a user definition, my aim is to rectify this omission and conduct a trial of the patch. This testing phase is crucial to ensure the patch's efficacy prior to the submission of a pull request.

Your insights or resource recommendations to aid in this process would be highly valued.

Thank you for your anticipated assistance!

      using Microsoft.SemanticKernel.Planning;
       // Model file /Users/uu/Downloads/Phi-3-mini-128k-instruct.Q8_0.gguf
       // Model API server cmd:  python3 -m llama_cpp.server    
       // Connect to the Phi-3-mini-128k-instruct as OpenAI 
        var ModelUrl = "http://127.0.0.1:8000";
        var ModelName = "phi3q8"; 

        var builder = Kernel.CreateBuilder()
            .AddOpenAIChatCompletion(
            modelId: ModelName,
            endpoint: new Uri(ModelUrl),
            apiKey: "api"
        );

        Kernel kernel = builder.Build();
        builder.Plugins.AddFromType<GitPlugin>(); // git clone
        builder.Plugins.AddFromType<FileIOPlugin>(); // read the file content
        builder.Plugins.AddFromType<DockerfileAnalyzePlugin>(); // analyze the  Dockerfile content if it requires modification
        var planner = new HandlebarsPlanner(new HandlebarsPlannerOptions {
            AllowLoops = false,
            ExecutionSettings = new OpenAIPromptExecutionSettings()
            {
                //ChatSystemPrompt = instructions,
                ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions,
                Temperature = 0.2
            },

        });

            request = $"""
                Attempt to clone the git repository from the specified URL {repoUrl} and branch {repoBranch} into the directory {repoPath}.
                If the cloning process fails, immediately stop and write "Error during cloning: [actual error message]".
                If the cloning is successful and the output confirms "Successfully Cloned", proceed to the next step.
                Navigate to the directory {repoPath} and attempt to read the Dockerfile located at {analyzeFilePath}.
                If reading the Dockerfile fails, immediately stop and write "Error during reading Dockerfile: [actual error message]".
                If the Dockerfile is read successfully, analyze its contents.
                Write down the results of the analysis.
                If the analysis process fails, immediately stop and write "Error during analysis: [actual error message]".
                Please ensure to replace the placeholders with the actual values before executing the prompt

                """;
try
{ 
      plan = await planner.CreatePlanAsync(kernel, request); catch (PlanCreationException e)
{
    await Task.Delay(1000); // Wait for a second before retrying
    var errorDetails = e.InnerException?.Message;
    var promptDetails = e.CreatePlanPrompt;
    var modelResults = e.ModelResults?.Content; // Proposed plan

    Console.WriteLine($"Plan creation failed with error: {errorDetails}");
    Console.WriteLine($"CreatePlan Prompt: {promptDetails}");
    Console.WriteLine($"Model Results: {modelResults}");
}
catch (PlanCreationException e)
{
    var errorDetails = e.InnerException?.Message;
    var promptDetails = e.CreatePlanPrompt;
    var modelResults = e.ModelResults?.Content; // Proposed plan

    Console.WriteLine($"Plan creation failed with error: {errorDetails}");
    Console.WriteLine($"CreatePlan Prompt: {promptDetails}");
    Console.WriteLine($"Model Results: {modelResults}");
}
catch (Exception ex)
{
    Console.WriteLine($"An error occurred: {ex.Message}. Retrying...");
}

The errors:

Plan creation failed with error: [InvalidTemplate] Identified multiple Handlebars templates in model response. Please try again.

Model Results:

1. Use {{GitPlugin-CloneRepository}} helper with the following inputs: repositoryUrl="https://u.visualstudio.com/DefaultCollection/XX/_git/client", path="/Users/uu/GIT/client".
2. Check if the output equals "Successfully Cloned". If it does, proceed to next steps. Otherwise, stop and report unsuccessful cloning.
3. Navigate to the directory Users/uu/GIT/client using {{set dir path}} helper.
4. Use {{FileIOPlugin-Read}} with inputs: path="/Users/uu/GIT/client/Dockerfile", content="". If this action is successful, proceed to the next step. Otherwise, stop and report unsuccessful reading of Dockerfile.
5. Invoke DockerAnalysisPlugin-AnalyzeDockerfile helper with input: path="/Users/uu/GIT/client/Dockerfile" and write down its output.

{{!-- Step 0: Clone the repository --}}
{{#if (try (GitPlugin-CloneRepository "https://uu.visualstudio.com/DefaultCollection/U/_git/client" "/Users/uu/GIT/client" "main"))}}
  {{!-- If cloning is successful, print the success message --}}
  {{json (concat "Successfully Cloned!")}}
{{else}}
  {{-- If cloning fails, stop and write error message --}}
  {{json (concat "Error during cloning: ", var)}}
{{/if}}

{{!-- Step 1: Navigate to the directory --}}
{{#try (cd "/Users/uu/GIT/client")}}
  {{-- If navigation fails, stop and write error message --}}
  {{json (concat "Error during reading Dockerfile: ", var)}}
{{else}}
  {{-- If navigation is successful, proceed to read the Dockerfile --}}
  {{!-- Step 2: Read the Dockerfile --}}
  {{#if (try (FileIOPlugin-Read "/Users/uu/GIT/client/Dockerfile"))}}
    {{-- If reading is successful, proceed to analyze the Dockerfile --}}
    {{!-- Step 3: Analyze the Dockerfile --}}
    {{#try (DockerAnalysisPlugin-AnalyzeDockerfile "/Users/uu/GIT/client/Dockerfile")}}
      {{-- If analysis is successful, print the results --}}
      {{json (concat "The Dockerfile was successfully analyzed. Results: ", var)}}
    {{else}}
      {{-- If analysis fails, stop and write error message --}}
      {{json (concat "Error during analysis: ", var)}}
    {{/try}}
  {{else}}
    {{-- If reading the Dockerfile fails, stop and write error message --}}
    {{json (concat "Error during reading Dockerfile: ", var)}}
  {{/if}}
{{/try}}

======== CreatePlan Prompt ========
An error occurred: Handlebars statement was not reduced to a single expression. Retrying...

If I use AzureOpenAI with GPT4- it works

builder = Kernel.CreateBuilder()
                .AddAzureOpenAIChatCompletion(
                    deploymentName: "gpt4",
                    modelId: ModelName,
                    endpoint: ModelUrl,
                    apiKey: ApiKey
                );
builder.Plugins.AddFromType<GitPlugin>();
builder.Plugins.AddFromType<FileIOPlugin>();
builder.Plugins.AddFromType<DockerAnalysisPlugin>();
var planner = new HandlebarsPlanner(new HandlebarsPlannerOptions {
            AllowLoops = false,
            ExecutionSettings = new OpenAIPromptExecutionSettings()
            {
                //ChatSystemPrompt = instructions,
                ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions,
                Temperature = 0.2
            },
});
string request = $"""
                Attempt to clone the git repository from the specified URL {repoUrl} and branch {repoBranch} into the directory {repoPath}.
                If the cloning process fails, immediately stop and write "Error during cloning: [actual error message]".
                If the cloning is successful and the output confirms "Successfully Cloned", proceed to the next step.
                Navigate to the directory {repoPath} and attempt to read the Dockerfile located at {analyzeFilePath}.
                If reading the Dockerfile fails, immediately stop and write "Error during reading Dockerfile: [actual error message]".
                If the Dockerfile is read successfully, analyze its contents.
                Write down the results of the analysis.
                If the analysis process fails, immediately stop and write "Error during analysis: [actual error message]".
                Please ensure to replace the placeholders with the actual values before executing the prompt

                """;

KernelArguments arguments = new KernelArguments
            {
                ["repositoryUrl"] = repoUrl,
                ["path"] = repoPath,
                ["branch"] = repoBranch,
                ["analyzeFilePath"] = analyzeFilePath,
                ["dockerSkipMsg"] = DockerAnalysisPlugin.ANALIS_OK

            };
plan = await planner.CreatePlanAsync(kernel, request);
Console.WriteLine($"Generated plan for request:\n{request}\n\n");
Console.WriteLine(plan);
Console.WriteLine("\n======== CreatePlan Prompt ========");
var planResult = await plan.InvokeAsync(kernel, arguments);
Console.WriteLine($"Plan Results: {planResult}");

Generated plan for request:

Attempt to clone the git repository from the specified URL https://uu.visualstudio.com/DefaultCollection/U/_git/client and branch main into the directory /Users/uu/GIT/client.
If the cloning process fails, immediately stop and write "Error during cloning: [actual error message]".
If the cloning is successful and the output confirms "Successfully Cloned", proceed to the next step.
Navigate to the directory /Users/uu/GIT/client and attempt to read the Dockerfile located at /Users/uu/GIT/client/Dockerfile.
If reading the Dockerfile fails, immediately stop and write "Error during reading Dockerfile: [actual error message]".
If the Dockerfile is read successfully, analyze its contents.
Write down the results of the analysis.
If the analysis process fails, immediately stop and write "Error during analysis: [actual error message]".
Please ensure to replace the placeholders with the actual values before executing the prompt



{{!-- Step 0: Extract key values --}}
{{set "repositoryUrl" "https://uu.visualstudio.com/DefaultCollection/U/_git/client"}}
{{set "branch" "main"}}
{{set "path" "/Users/uu/GIT/client"}}
{{set "dockerfilePath" "/Users/uu/GIT/client/Dockerfile"}}

{{!-- Step 1: Clone the Git repository --}}
{{#with (GitPlugin-CloneRepository repositoryUrl=repositoryUrl path=path branch=branch) as |cloneResult|}}
  {{#unless (equals cloneResult "Successfully Cloned")}}
    {{json (concat "Error during cloning: " cloneResult)}}
  {{else}}
    {{!-- Step 2: Read the Dockerfile --}}
    {{#with (FileIOPlugin-Read path=dockerfilePath) as |dockerfileContent|}}
      {{#unless dockerfileContent}}
        {{json (concat "Error during reading Dockerfile: " dockerfileContent)}}
      {{else}}
        {{!-- Step 3: Analyze the Dockerfile --}}
        {{#with (DockerAnalysisPlugin-AnalyzeDockerfile path=dockerfilePath) as |analysisResult|}}
          {{#unless analysisResult}}
            {{json (concat "Error during analysis: " analysisResult)}}
          {{else}}
            {{!-- Step 4: Write the results of the analysis --}}
            {{json analysisResult}}
          {{/unless}}
        {{/with}}
      {{/unless}}
    {{/with}}
  {{/unless}}
{{/with}}

[sophialagerkranspandey]

To proceed with helping you we'd like to understand if you're wanting to create a plan and look at a plan or manually check off a plan before it's run? Thanks!

[weldpua2008]

Hey @sophialagerkranspandey
I am seeking to understand the sequence of events as a new user of Semantic Kernel. Here are my questions:

• Is the error I'm encountering due to the PHi3 model generating incorrect syntax, resulting in the "An error occurred: Handlebars statement was not reduced to a single expression" message? If so, which component is responsible for this error? Is it the C# Handlebars library?

• Why did this error occur with the PHi-3 model and not with GPT-4?

• Could it be related to preview versions of the packages?

<ItemGroup>
    <PackageReference Include="Microsoft.SemanticKernel" Version="1.14.1" />
    <PackageReference Include="OllamaSharp" Version="2.0.1" />
    <PackageReference Include="Microsoft.SemanticKernel.Planners.Handlebars" Version="1.13.0-preview" />
    <PackageReference Include="Microsoft.SemanticKernel.Plugins.Document" Version="1.13.0-alpha" />
    <PackageReference Include="Microsoft.SemanticKernel.Plugins.Web" Version="1.13.0-alpha" />
    <PackageReference Include="Microsoft.SemanticKernel.Plugins.Memory" Version="1.13.0-alpha" />
    <PackageReference Include="Microsoft.SemanticKernel.Plugins.Core" Version="1.13.0-alpha" />
    <PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="8.0.0" />
  </ItemGroup>

• Is the Semantic Kernel Planner designed exclusively for creating plans with LLMs in advance and then executing those pre-created plans? Or have I misunderstood the Planner's functionality, and it is actually capable of iterating and executing tasks until reaching the goal?
• Where can I find the list of supported syntax for Handlebars templating?

Thank you for your assistance.

[sophialagerkranspandey]

The Semantic Kernel Planner is not created exclusively for LLMs. Depending on how much training the model has with handlebars templating - the response will vary.