|
| 1 | +--- |
| 2 | +meta: |
| 3 | + title: How to use function calling |
| 4 | + description: Learn how to implement function calling capabilities using Scaleway's Chat Completions API service. |
| 5 | +content: |
| 6 | + h1: How to use function calling |
| 7 | + paragraph: Learn how to enhance AI interactions by integrating external tools and functions using Scaleway's Chat Completions API service. |
| 8 | +tags: chat-completions-api |
| 9 | +dates: |
| 10 | + validation: 2024-09-24 |
| 11 | + posted: 2024-09-24 |
| 12 | +--- |
| 13 | + |
| 14 | +Scaleway's Chat Completions API supports function calling as introduced by OpenAI. |
| 15 | + |
| 16 | +## What is function calling? |
| 17 | + |
| 18 | +Function calling allows a large language model (LLM) to interact with external tools or APIs, executing specific tasks based on user requests. The LLM identifies the appropriate function, extracts the required parameters, and returns the tool call to be done as structured data, typically in JSON format. While errors can occur, custom parsers or tools like LlamaIndex and LangChain can help ensure valid results. |
| 19 | + |
| 20 | +<Macro id="requirements" /> |
| 21 | + |
| 22 | +- Access to Generative APIs. |
| 23 | +- [Owner](/identity-and-access-management/iam/concepts/#owner) status or [IAM permissions](/identity-and-access-management/iam/concepts/#permission) allowing you to perform actions in the intended Organization |
| 24 | +- A valid [API key](/identity-and-access-management/iam/how-to/create-api-keys/) for API authentication |
| 25 | +- Python 3.7+ installed on your system |
| 26 | + |
| 27 | +## Supported models |
| 28 | + |
| 29 | +* llama-3.1-8b-instruct |
| 30 | +* llama-3.1-70b-instruct |
| 31 | +* mistral-nemo-instruct-2407 |
| 32 | + |
| 33 | +## Understanding function calling |
| 34 | + |
| 35 | +Function calling consists of three main components: |
| 36 | +- **Tool definitions**: JSON schemas that describe available functions and their parameters |
| 37 | +- **Tool selection**: Automatic or manual selection of appropriate functions based on user queries |
| 38 | +- **Tool execution**: Processing function calls and handling their responses |
| 39 | + |
| 40 | +The workflow typically follows these steps: |
| 41 | +1. Define available tools using JSON schema |
| 42 | +2. Send system and user query along with tool definitions |
| 43 | +3. Process model's function selection |
| 44 | +4. Execute selected functions |
| 45 | +5. Return results to model for final response |
| 46 | + |
| 47 | +## Code examples |
| 48 | + |
| 49 | +<Message type="tip"> |
| 50 | + Before diving into the code examples, ensure you have the necessary libraries installed: |
| 51 | + ```bash |
| 52 | + pip install openai |
| 53 | + ``` |
| 54 | +</Message> |
| 55 | + |
| 56 | +We will demonstrate function calling using a flight scheduling system that allows users to check available flights between European airports. |
| 57 | + |
| 58 | +### Basic function definition |
| 59 | + |
| 60 | +First, let's define our flight schedule function and its schema: |
| 61 | + |
| 62 | +```python |
| 63 | +from openai import OpenAI |
| 64 | +import json |
| 65 | + |
| 66 | +def get_flight_schedule(departure_airport: str, destination_airport: str, departure_date: str) -> dict: |
| 67 | + """ |
| 68 | + Retrieves flight schedules between two European airports on a specific date. |
| 69 | + """ |
| 70 | + # Mock flight schedule data |
| 71 | + flights = { |
| 72 | + "CDG-LHR-2024-11-01": [ |
| 73 | + {"flight_number": "AF123", "airline": "Air France", "departure_time": "08:00", "arrival_time": "09:00"}, |
| 74 | + {"flight_number": "BA456", "airline": "British Airways", "departure_time": "10:00", "arrival_time": "11:00"}, |
| 75 | + {"flight_number": "LH789", "airline": "Lufthansa", "departure_time": "14:00", "arrival_time": "15:00"} |
| 76 | + ], |
| 77 | + "AMS-MUC-2024-11-01": [ |
| 78 | + {"flight_number": "KL101", "airline": "KLM", "departure_time": "07:30", "arrival_time": "09:00"}, |
| 79 | + {"flight_number": "LH202", "airline": "Lufthansa", "departure_time": "12:00", "arrival_time": "13:30"} |
| 80 | + ] |
| 81 | + } |
| 82 | + |
| 83 | + key = f"{departure_airport}-{destination_airport}-{departure_date}" |
| 84 | + return flights.get(key, {"error": "No flights found for this route and date."}) |
| 85 | + |
| 86 | +# Define the tool specification |
| 87 | +tools = [{ |
| 88 | + "type": "function", |
| 89 | + "function": { |
| 90 | + "name": "get_flight_schedule", |
| 91 | + "description": "Get available flights between two European airports on a specific date", |
| 92 | + "parameters": { |
| 93 | + "type": "object", |
| 94 | + "properties": { |
| 95 | + "departure_airport": { |
| 96 | + "type": "string", |
| 97 | + "description": "The IATA code of the departure airport (e.g., CDG, LHR)" |
| 98 | + }, |
| 99 | + "destination_airport": { |
| 100 | + "type": "string", |
| 101 | + "description": "The IATA code of the destination airport" |
| 102 | + }, |
| 103 | + "departure_date": { |
| 104 | + "type": "string", |
| 105 | + "description": "The date of departure in YYYY-MM-DD format" |
| 106 | + } |
| 107 | + }, |
| 108 | + "required": ["departure_airport", "destination_airport", "departure_date"] |
| 109 | + } |
| 110 | + } |
| 111 | +}] |
| 112 | +``` |
| 113 | + |
| 114 | +### Simple function call example |
| 115 | + |
| 116 | +Here is how to implement a basic function call: |
| 117 | + |
| 118 | +```python |
| 119 | +# Initialize the OpenAI client |
| 120 | +client = OpenAI( |
| 121 | + base_url="https://api.scaleway.ai/v1", |
| 122 | + api_key="<YOUR_API_KEY>" |
| 123 | +) |
| 124 | + |
| 125 | +# Create a simple query |
| 126 | +messages = [ |
| 127 | + { |
| 128 | + "role": "system", |
| 129 | + "content": "You are a helpful flight assistant." |
| 130 | + }, |
| 131 | + { |
| 132 | + "role": "user", |
| 133 | + "content": "What flights are available from CDG to LHR on November 1st, 2024?" |
| 134 | + } |
| 135 | +] |
| 136 | + |
| 137 | +# Make the API call |
| 138 | +response = client.chat.completions.create( |
| 139 | + model="llama-3.1-70b-instruct", |
| 140 | + messages=messages, |
| 141 | + tools=tools, |
| 142 | + tool_choice="auto" |
| 143 | +) |
| 144 | +``` |
| 145 | + |
| 146 | +<Message type="note"> |
| 147 | + The model automatically decides which functions to call. However, you can specify a particular function by using the `tool_choice` parameter. In the example above, you can replace `tool_choice=auto` with `tool_choice={"type": "function", "function": {"name": "get_flight_schedule"}}` to explicitly call the desired function. |
| 148 | +</Message> |
| 149 | + |
| 150 | +### Multi-turn conversation handling |
| 151 | + |
| 152 | +For more complex interactions, you will need to handle multiple turns of conversation: |
| 153 | + |
| 154 | +```python |
| 155 | +# Process the tool call |
| 156 | +if response.choices[0].message.tool_calls: |
| 157 | + tool_call = response.choices[0].message.tool_calls[0] |
| 158 | + |
| 159 | + # Execute the function |
| 160 | + if tool_call.function.name == "get_flight_schedule": |
| 161 | + function_args = json.loads(tool_call.function.arguments) |
| 162 | + function_response = get_flight_schedule(**function_args) |
| 163 | + |
| 164 | + # Add results to the conversation |
| 165 | + messages.extend([ |
| 166 | + { |
| 167 | + "role": "assistant", |
| 168 | + "content": None, |
| 169 | + "tool_calls": [tool_call] |
| 170 | + }, |
| 171 | + { |
| 172 | + "role": "tool", |
| 173 | + "name": tool_call.function.name, |
| 174 | + "content": json.dumps(function_response), |
| 175 | + "tool_call_id": tool_call.id |
| 176 | + } |
| 177 | + ]) |
| 178 | + |
| 179 | + # Get final response |
| 180 | + final_response = client.chat.completions.create( |
| 181 | + model="llama-3.1-70b-instruct", |
| 182 | + messages=messages |
| 183 | + ) |
| 184 | + print(final_response.choices[0].message.content) |
| 185 | +``` |
| 186 | + |
| 187 | +### Parallel function calling |
| 188 | + |
| 189 | +<Message type="important"> |
| 190 | + Meta models do not support parallel tool calls. |
| 191 | +</Message> |
| 192 | + |
| 193 | +In addition to one function call described above, you can also call multiple functions in a single turn. |
| 194 | +This section shows an example for how you can use parallel function calling. |
| 195 | + |
| 196 | +Define the tools: |
| 197 | + |
| 198 | +``` |
| 199 | +def open_floor_space(floor_number: int) -> bool: |
| 200 | + """Opens up the specified floor for party space by unlocking doors and moving furniture.""" |
| 201 | + print(f"Floor {floor_number} is now open party space!") |
| 202 | + return True |
| 203 | +
|
| 204 | +def set_lobby_vibe(party_mode: bool) -> str: |
| 205 | + """Switches lobby screens and lighting to party mode.""" |
| 206 | + status = "party mode activated!" if party_mode else "back to business mode" |
| 207 | + print(f"Lobby is now in {status}") |
| 208 | + return "The lobby is ready to party!" |
| 209 | +
|
| 210 | +def prep_snack_station(activate: bool) -> bool: |
| 211 | + """Converts the cafeteria into a snack and drink station.""" |
| 212 | + print(f"Snack station is {'open and stocked!' if activate else 'closed.'}") |
| 213 | + return True |
| 214 | +``` |
| 215 | + |
| 216 | +Define the specifications: |
| 217 | + |
| 218 | +``` |
| 219 | +tools = [ |
| 220 | + { |
| 221 | + "type": "function", |
| 222 | + "function": { |
| 223 | + "name": "open_floor_space", |
| 224 | + "description": "Opens up an entire floor for the party", |
| 225 | + "parameters": { |
| 226 | + "type": "object", |
| 227 | + "properties": { |
| 228 | + "floor_number": { |
| 229 | + "type": "integer", |
| 230 | + "description": "Which floor to open up" |
| 231 | + } |
| 232 | + }, |
| 233 | + "required": ["floor_number"] |
| 234 | + } |
| 235 | + } |
| 236 | + }, |
| 237 | + { |
| 238 | + "type": "function", |
| 239 | + "function": { |
| 240 | + "name": "set_lobby_vibe", |
| 241 | + "description": "Transform lobby atmosphere into party mode", |
| 242 | + "parameters": { |
| 243 | + "type": "object", |
| 244 | + "properties": { |
| 245 | + "party_mode": { |
| 246 | + "type": "boolean", |
| 247 | + "description": "True for party, False for business" |
| 248 | + } |
| 249 | + }, |
| 250 | + "required": ["party_mode"] |
| 251 | + } |
| 252 | + } |
| 253 | + }, |
| 254 | + { |
| 255 | + "type": "function", |
| 256 | + "function": { |
| 257 | + "name": "prep_snack_station", |
| 258 | + "description": "Set up the snack and drink station", |
| 259 | + "parameters": { |
| 260 | + "type": "object", |
| 261 | + "properties": { |
| 262 | + "activate": { |
| 263 | + "type": "boolean", |
| 264 | + "description": "True to open, False to close" |
| 265 | + } |
| 266 | + }, |
| 267 | + "required": ["activate"] |
| 268 | + } |
| 269 | + } |
| 270 | + } |
| 271 | +] |
| 272 | +``` |
| 273 | + |
| 274 | +Next, call the model with proper instructions |
| 275 | + |
| 276 | +``` |
| 277 | +system_prompt = """ |
| 278 | +You are an office party control assistant. When asked to transform the office into a party space, you should: |
| 279 | +1. Open up a floor for the party |
| 280 | +2. Transform the lobby into party mode |
| 281 | +3. Set up the snack station |
| 282 | +Make all these changes at once for an instant office party! |
| 283 | +""" |
| 284 | +
|
| 285 | +messages = [ |
| 286 | + {"role": "system", "content": system_prompt}, |
| 287 | + {"role": "user", "content": "Turn this office building into a party!"} |
| 288 | +] |
| 289 | +``` |
| 290 | + |
| 291 | +## Best practices |
| 292 | + |
| 293 | +When implementing function calling, follow these guidelines for optimal results: |
| 294 | + |
| 295 | +1. **Function design** |
| 296 | + - Keep function names clear and descriptive |
| 297 | + - Limit the number of functions to 7 or fewer per conversation |
| 298 | + - Use detailed parameter descriptions in your JSON schema |
| 299 | + |
| 300 | +2. **Parameter handling** |
| 301 | + - Always specify required parameters |
| 302 | + - Use appropriate data types and validation |
| 303 | + - Include example values in parameter descriptions |
| 304 | + |
| 305 | +3. **Error handling** |
| 306 | + - Implement robust error handling for function execution |
| 307 | + - Return clear error messages that the model can interpret |
| 308 | + - Handle edge cases gracefully |
| 309 | + |
| 310 | +4. **Performance optimization** |
| 311 | + - Set appropriate temperature values (lower for more precise function calls) |
| 312 | + - Cache frequently accessed data when possible |
| 313 | + - Minimize the number of turns in multi-turn conversations |
| 314 | + |
| 315 | +<Message type="note"> |
| 316 | + For production applications, always implement proper error handling and input validation. The examples above focus on the happy path for clarity. |
| 317 | +</Message> |
| 318 | + |
| 319 | +## Further resources |
| 320 | + |
| 321 | +For more information about function calling and advanced implementations, refer to these resources: |
| 322 | + |
| 323 | +- [OpenAI Function Calling Guide](https://platform.openai.com/docs/guides/function-calling) |
| 324 | +- [JSON Schema Specification](https://json-schema.org/specification) |
| 325 | +- [Chat Completions API Reference](/ai-data/generative-apis/api-cli/using-chat-api/) |
| 326 | + |
| 327 | +Function calling significantly extends the capabilities of language models by allowing them to interact with external tools and APIs. |
| 328 | + |
| 329 | +<Message type="note"> |
| 330 | + We can't wait to see what you will build with function calls. Tell us what you are up to, share your experiments on Scaleway's [Slack community](https://slack.scaleway.com/) #ai |
| 331 | +</Message> |
0 commit comments