How would you implement your own function calling?

[dotMorten]

I've built my own IChatCompletionService for the Phi4 model and this works great. I've also managed to supply it with function call signatures and get it to respond with function call requests. However these just come back as streaming text that I can't really tell apart from the normal text responses, instead of a response with a specific tool_call role.
How would one go about implementing function calling in a customer provider in C#? I've browsed the source code here and find bits and pieces but can't really make too much sense of it. Any pointers and examples/breakdown of existing classes would be super helpful.

Here's what I've sort of played with, but not sure if this is the right place to plug things in IChatCompletionService.GetStreamingChatMessageContentsAsync, or how to really move on from here:

                using var generator = new Generator(model, generatorParams);
                bool isToolCall = false;
                string toolcall = "";
                while (!generator.IsDone())
                {
                    // generator.ComputeLogits();
                    generator.GenerateNextToken();
                    var seq = generator.GetSequence(0)[^1];
                    var str = tokenizerStream.Decode(seq);

                    if (cancellationToken.IsCancellationRequested)
                        break;

                    if (str == "<|tool_call|>")
                    {
                        isToolCall = true;
                        continue;
                    }
                    else if (str == "<|/tool_call|>")
                    {
                        // TODO: Call function somehow
                        isToolCall = false;
                        toolcall = "";
                        history.Add(new ChatMessageContent(AuthorRole.Tool, toolcall));
                        Debug.WriteLine("Tool call: " + toolcall);
                        continue;
                    }
                    if (isToolCall)
                    {
                        toolcall += str;
                    }
                    else
                    {
                        yield return str;
                    }
;
                    await Task.Yield();
                    if (cancellationToken.IsCancellationRequested)
                        break;
                }

[moonbox3]

Add @markwallace-microsoft for visibility