Add SLM Engine support function calling (#1582)

This pull request introduces significant enhancements to the SLM Engine,
focusing on adding function-calling support and improving the build
process for ONNX Runtime-GenAI. The most notable changes include the
implementation of function-calling capabilities, updates to the build
scripts to handle fallback modes, and the addition of new structures and
methods to support function tools in the C++ codebase.

### Function-Calling Support:
* Added a new section in `README.md` to document function-calling
capabilities, including key features, an example request, and structured
JSON output for function calls (`examples/slm_engine/README.md`).
* Introduced new structs (`FunctionParameter`, `FunctionTool`,
`FunctionCallOptions`, `FunctionCall`, and `FunctionCallResult`) to
represent function tools, parameters, and results in the SLM Engine
(`examples/slm_engine/src/cpp/slm_engine.h`).
* Added methods to parse tools from JSON, format input with tools,
create grammar for function calling, and handle function call results
(`examples/slm_engine/src/cpp/slm_engine.h`).
* Updated `InputDecoder` to handle tools and added a new `TOOL` role to
support function-calling scenarios
(`examples/slm_engine/src/cpp/input_decoder.cpp`,
`examples/slm_engine/src/cpp/input_decoder.h`).

### Build Process Improvements:
* Enhanced the build process for ONNX Runtime-GenAI to support guidance
mode for function calling and added a fallback mechanism if guidance
mode fails (`examples/slm_engine/build_scripts/build_deps.py`).
* Updated build scripts to include descriptive logging for
function-calling support
(`examples/slm_engine/build_scripts/build_deps.py`).

### Server Capabilities Update:
* Modified the SLM server to include "function_calling" as a capability
in its response, reflecting the new feature
(`examples/slm_engine/src/cpp/slm_server.cpp`).

These changes collectively enhance the SLM Engine's functionality,
making it more adaptable for use cases requiring intelligent function
invocation and robust build processes.

Author: kinfey

examples/slm_engine/README.md

@@ -320,6 +320,84 @@ The SLM server supports the following REST APIs (click to expand):
 
 </details>
 
+### Function Calling Support
+
+The SLM Engine enabling the model to intelligently select and invoke predefined functions based on user requests. This feature allows developers to extend the model's capabilities by providing custom tools and functions that the AI can use to perform specific tasks.
+
+#### Key Features:
+- **Tool Definition**: Define custom functions with parameters and descriptions
+- **Intelligent Function Selection**: The model automatically determines which function to call based on user input
+- **Structured Output**: Returns function calls in a standardized JSON format
+
+#### Example Function Calling Request
+
+The following example demonstrates how to use function calling with the SLM Engine for booking flights and hotels:
+
+```bash
+curl -X POST http://localhost:8000/completions -H "Content-Type: application/json" --data '{
+    "messages": [
+        {
+            "role": "system",
+            "content": "You are a helpful assistant with these tools."
+        },
+        {
+            "role": "user",
+            "content": "book flight ticket from Beijing to Paris(using airport code) in 2025-12-04 to 2025-12-10 , then book hotel from 2025-12-04 to 2025-12-10 in Paris"
+        }
+    ],
+    "tools": [
+        {
+            "name": "booking_flight_tickets", 
+            "description": "booking flights", 
+            "parameters": {
+                "origin_airport_code": {
+                    "description": "The name of Departure airport code", 
+                    "type": "string"
+                }, 
+                "destination_airport_code": {
+                    "description": "The name of Destination airport code", 
+                    "type": "string"
+                }, 
+                "departure_date": {
+                    "description": "The date of outbound flight", 
+                    "type": "string"
+                }, 
+                "return_date": {
+                    "description": "The date of return flight", 
+                    "type": "string"
+                }
+            }
+        }, 
+        {
+            "name": "booking_hotels", 
+            "description": "booking hotel", 
+            "parameters": {
+                "destination": {
+                    "description": "The name of the city", 
+                    "type": "string"
+                }, 
+                "check_in_date": {
+                    "description": "The date of check in", 
+                    "type": "string"
+                }, 
+                "checkout_date": {
+                    "description": "The date of check out", 
+                    "type": "string"
+                }
+            }
+        }
+    ],
+    "temperature": 0.00001,
+    "max_tokens": 4096,
+    "top_p": 1.0,
+    "do_sample": false
+}'
+```
+
+The model will analyze the user's request and generate appropriate function calls with the correct parameters, enabling seamless integration with external APIs and services.
+
+***Note*** - This time we just support Phi and Llama,Qwen3 model
+
 ### C++ Application using the SLMEngine
 
 The SLMEngine is designed to be used from another C++ application running on the Edge. Integrating the SLMEngine into another C++ project using cmake is illustrated below.

examples/slm_engine/build_scripts/build.py

@@ -70,7 +70,7 @@ def main():
         cmake_generator,
         TOPLEVEL_DIR,
         f"-DARTIFACTS_DIR={artifacts_dir}",
-        f"-DCMAKE_BUILD_TYPE={args.build_type}",
+        f"-DCMAKE_BUILD_TYPE={args.build_type}"
     ]
 
     # We keep the build directory prefix as same as that's returned by the

examples/slm_engine/build_scripts/build_deps.py

@@ -212,6 +212,7 @@ def build_ort(args, build_dir, artifacts_dir):
         "--parallel",
         "--config",
         args.build_type,
+        # "--use_guidance",
     ]
     if args.android:
         cmd_args.extend(
@@ -330,7 +331,7 @@ def build_ort_genai(args, artifacts_dir, ort_home):
         raise Exception("Failed to update submodules")
 
     # Now build the ORT-GenAI library
-    print(f"{MAGENTA}Building ONNX Runtime-GenAI{CLEAR}")
+    print(f"{MAGENTA}Building ONNX Runtime-GenAI with Guidance Support for Function Calling{CLEAR}")
     # Prepare the command arguments
     cmd_args = [
         "--skip_wheel",
@@ -340,6 +341,10 @@ def build_ort_genai(args, artifacts_dir, ort_home):
         args.build_type,
         "--cmake_extra_defines",
         "ENABLE_PYTHON=OFF",
+        # "USE_GUIDANCE=ON",
+        # "--use_guidance",  # Enable guidance support for constrained JSON generation
+        # Note: If Python linking issues occur, comment out --use_guidance above
+        # Function calling will work in both guidance and fallback modes
     ]
     if ort_home is None:
         raise Exception(
@@ -368,7 +373,20 @@ def build_ort_genai(args, artifacts_dir, ort_home):
     python_executable = sys.executable
     result = subprocess.call([python_executable, "build.py"] + cmd_args)
     if result != 0:
-        raise Exception(f"{RED}Failed to build ORT-GenAI{CLEAR}")
+        # If guidance build fails, try fallback mode
+        print(f"{RED}Guidance build failed. Attempting fallback mode without guidance...{CLEAR}")
+        # Remove --use_guidance from cmd_args
+        if "--use_guidance" in cmd_args:
+            cmd_args.remove("--use_guidance")
+        
+        print(f"{MAGENTA}Running build.py with fallback args: {cmd_args}{CLEAR}")
+        result = subprocess.call([python_executable, "build.py"] + cmd_args)
+        if result != 0:
+            raise Exception(f"{RED}Failed to build ORT-GenAI in both guidance and fallback modes{CLEAR}")
+        else:
+            print(f"{MAGENTA}Successfully built ORT-GenAI in fallback mode{CLEAR}")
+    else:
+        print(f"{MAGENTA}Successfully built ORT-GenAI with guidance support{CLEAR}")
 
     # Now install the ORT-GenAI library
     build_dir_name = f"build/{get_platform_dirname(args)}/{args.build_type}"

examples/slm_engine/src/VERSION.txt

@@ -1 +1 @@
-2.0.0
+3.0.0

examples/slm_engine/src/cpp/input_decoder.cpp

@@ -90,6 +90,12 @@ class OpenAIInputDecoder : public InputDecoder {
                << CLEAR << endl;
         }
       }
+
+      // Handle tools parameter for function calling
+      if (json_msg.contains("tools")) {
+        decoded_params.ToolsJson = json_msg["tools"].dump();
+        decoded_params.HasTools = true;
+      }
     } catch (json::parse_error& err) {
       cout << RED << "Error in JSON At: " << err.what() << CLEAR << endl;
       return false;

examples/slm_engine/src/cpp/input_decoder.h

@@ -24,14 +24,17 @@ class InputDecoder {
   struct InputParams {
     enum class Role { SYSTEM,
                       USER,
-                      ASSISTANT };
+                      ASSISTANT,
+                      TOOL };
 
     // Utility function to convert string to Role
     static Role ToRole(const std::string& role) {
       if (role == "system") {
         return Role::SYSTEM;
       } else if (role == "user") {
         return Role::USER;
+      } else if (role == "tool") {
+        return Role::TOOL;
       } else {
         return Role::ASSISTANT;
       }
@@ -51,11 +54,16 @@ class InputDecoder {
     float TopP;
     uint32_t TopK;
 
+    // Function calling support
+    std::string ToolsJson;  // Raw tools JSON string from input
+    bool HasTools;
+
     explicit InputParams() {
       MaxGeneratedTokens = 512;
       Temperature = 0.00000000000001f;
       TopK = 50;
       TopP = 1.0f;
+      HasTools = false;
     }
 
     std::string get_messages() {
@@ -68,6 +76,9 @@ class InputDecoder {
           case Role::USER:
             output << "{\"role\": \"user\", ";
             break;
+          case Role::TOOL:
+            output << "{\"role\": \"tool\", ";
+            break;
           case Role::ASSISTANT:
             output << "{\"role\": \"assistant\", ";
             break;
@@ -89,6 +100,9 @@ class InputDecoder {
           case Role::USER:
             output << "USER";
             break;
+          case Role::TOOL:
+            output << "TOOL";
+            break;
           case Role::ASSISTANT:
             output << "ASSISTANT";
             break;