Skip to content

Commit 9997fb6

Browse files
authored
docs(toolbox-core): Update README to include guidance on session lifecycle management (#259)
* docs: Update README to include guidance on session lifecycle management * docs: Improve README * docs: Improve README * docs: Improve README * docs: Improve README * docs: Add client lifecycle management for sync client * docs: Improve README * docs: Improve the README doc
1 parent 9400621 commit 9997fb6

File tree

1 file changed

+111
-77
lines changed

1 file changed

+111
-77
lines changed

packages/toolbox-core/README.md

Lines changed: 111 additions & 77 deletions
Original file line numberDiff line numberDiff line change
@@ -21,31 +21,31 @@ involving Large Language Models (LLMs).
2121
- [Quickstart](#quickstart)
2222
- [Usage](#usage)
2323
- [Loading Tools](#loading-tools)
24-
- [Load a toolset](#load-a-toolset)
25-
- [Load a single tool](#load-a-single-tool)
24+
- [Load a toolset](#load-a-toolset)
25+
- [Load a single tool](#load-a-single-tool)
2626
- [Invoking Tools](#invoking-tools)
2727
- [Synchronous Usage](#synchronous-usage)
2828
- [Use with LangGraph](#use-with-langgraph)
29-
- [Authenticating to the Toolbox Server](#authenticating-to-the-toolbox-server)
30-
- [When is Client-to-Server Authentication Needed?](#when-is-client-to-server-authentication-needed)
31-
- [How it works](#how-it-works)
32-
- [Configuration](#configuration)
33-
- [Authenticating with Google Cloud Servers](#authenticating-with-google-cloud-servers)
34-
- [Step by Step Guide for Cloud Run](#step-by-step-guide-for-cloud-run)
29+
- [Client to Server Authentication](#client-to-server-authentication)
30+
- [When is Client-to-Server Authentication Needed?](#when-is-client-to-server-authentication-needed)
31+
- [How it works](#how-it-works)
32+
- [Configuration](#configuration)
33+
- [Authenticating with Google Cloud Servers](#authenticating-with-google-cloud-servers)
34+
- [Step by Step Guide for Cloud Run](#step-by-step-guide-for-cloud-run)
3535
- [Authenticating Tools](#authenticating-tools)
36-
- [When is Authentication Needed?](#when-is-authentication-needed)
37-
- [Supported Authentication Mechanisms](#supported-authentication-mechanisms)
38-
- [Step 1: Configure Tools in Toolbox Service](#step-1-configure-tools-in-toolbox-service)
39-
- [Step 2: Configure SDK Client](#step-2-configure-sdk-client)
40-
- [Provide an ID Token Retriever Function](#provide-an-id-token-retriever-function)
41-
- [Option A: Add Authentication to a Loaded Tool](#option-a-add-authentication-to-a-loaded-tool)
42-
- [Option B: Add Authentication While Loading Tools](#option-b-add-authentication-while-loading-tools)
43-
- [Complete Authentication Example](#complete-authentication-example)
36+
- [When is Authentication Needed?](#when-is-authentication-needed)
37+
- [Supported Authentication Mechanisms](#supported-authentication-mechanisms)
38+
- [Step 1: Configure Tools in Toolbox Service](#step-1-configure-tools-in-toolbox-service)
39+
- [Step 2: Configure SDK Client](#step-2-configure-sdk-client)
40+
- [Provide an ID Token Retriever Function](#provide-an-id-token-retriever-function)
41+
- [Option A: Add Authentication to a Loaded Tool](#option-a-add-authentication-to-a-loaded-tool)
42+
- [Option B: Add Authentication While Loading Tools](#option-b-add-authentication-while-loading-tools)
43+
- [Complete Authentication Example](#complete-authentication-example)
4444
- [Binding Parameter Values](#binding-parameter-values)
45-
- [Why Bind Parameters?](#why-bind-parameters)
46-
- [Option A: Binding Parameters to a Loaded Tool](#option-a-binding-parameters-to-a-loaded-tool)
47-
- [Option B: Binding Parameters While Loading Tools](#option-b-binding-parameters-while-loading-tools)
48-
- [Binding Dynamic Values](#binding-dynamic-values)
45+
- [Why Bind Parameters?](#why-bind-parameters)
46+
- [Option A: Binding Parameters to a Loaded Tool](#option-a-binding-parameters-to-a-loaded-tool)
47+
- [Option B: Binding Parameters While Loading Tools](#option-b-binding-parameters-while-loading-tools)
48+
- [Binding Dynamic Values](#binding-dynamic-values)
4949
- [Contributing](#contributing)
5050
- [License](#license)
5151
- [Support](#support)
@@ -69,6 +69,14 @@ pip install toolbox-core
6969
> - If you prefer synchronous execution, refer to the [Synchronous
7070
> Usage](#synchronous-usage) section below.
7171
72+
> [!IMPORTANT]
73+
>
74+
> The `ToolboxClient` (and its synchronous counterpart `ToolboxSyncClient`)
75+
> interacts with network resources using an underlying HTTP client session. You
76+
> should remember to use a context manager or explicitly call `close()` to clean
77+
> up these resources. If you provide your own session, you'll need to close it
78+
> in addition to calling `ToolboxClient.close()`.
79+
7280
## Quickstart
7381

7482
Here's a minimal example to get you started. Ensure your Toolbox service is
@@ -80,15 +88,29 @@ from toolbox_core import ToolboxClient
8088

8189
async def main():
8290
# Replace with the actual URL where your Toolbox service is running
83-
toolbox = ToolboxClient("http://127.0.0.1:5000")
84-
weather_tool = await toolbox.load_tool("get_weather")
85-
result = await weather_tool(location="London")
86-
print(result)
91+
async with ToolboxClient("http://127.0.0.1:5000") as toolbox:
92+
weather_tool = await toolbox.load_tool("get_weather")
93+
result = await weather_tool(location="London")
94+
print(result)
8795

8896
if __name__ == "__main__":
8997
asyncio.run(main())
9098
```
9199

100+
> [!IMPORTANT]
101+
> If you initialize `ToolboxClient` without providing an external session and
102+
> cannot use `async with`, you must explicitly close the client using `await
103+
> toolbox.close()` in a `finally` block. This ensures the internally created
104+
> session is closed.
105+
>
106+
> ```py
107+
> toolbox = ToolboxClient("http://127.0.0.1:5000")
108+
> try:
109+
> # ... use toolbox ...
110+
> finally:
111+
> await toolbox.close()
112+
> ```
113+
92114
## Usage
93115
94116
Import and initialize a Toolbox client, pointing it to the URL of your running
@@ -98,11 +120,23 @@ Toolbox service.
98120
from toolbox_core import ToolboxClient
99121
100122
# Replace with your Toolbox service's URL
101-
toolbox = ToolboxClient("http://127.0.0.1:5000")
123+
async with ToolboxClient("http://127.0.0.1:5000") as toolbox:
102124
```
103125
104126
All interactions for loading and invoking tools happen through this client.
105127

128+
> [!NOTE]
129+
> For advanced use cases, you can provide an external `aiohttp.ClientSession`
130+
> during initialization (e.g., `ToolboxClient(url, session=my_session)`). If you
131+
> provide your own session, you are responsible for managing its lifecycle;
132+
> `ToolboxClient` *will not* close it.
133+
134+
> [!IMPORTANT]
135+
> Closing the `ToolboxClient` also closes the underlying network session shared by
136+
> all tools loaded from that client. As a result, any tool instances you have
137+
> loaded will cease to function and will raise an error if you attempt to invoke
138+
> them after the client is closed.
139+
106140
## Loading Tools
107141

108142
You can load tools individually or in groups (toolsets) as defined in your
@@ -161,10 +195,10 @@ The `ToolboxSyncClient` handles communication with the Toolbox service synchrono
161195
```py
162196
from toolbox_core import ToolboxSyncClient
163197

164-
toolbox = ToolboxSyncClient("http://127.0.0.1:5000")
165-
weather_tool = toolbox.load_tool("get_weather")
166-
result = weather_tool(location="Paris")
167-
print(result)
198+
with ToolboxSyncClient("http://127.0.0.1:5000") as toolbox:
199+
weather_tool = toolbox.load_tool("get_weather")
200+
result = weather_tool(location="Paris")
201+
print(result)
168202
```
169203

170204
> [!TIP]
@@ -199,33 +233,33 @@ from langgraph.graph import StateGraph, MessagesState, START, END
199233
from langgraph.prebuilt import ToolNode
200234
from langchain.tools import StructuredTool
201235

202-
toolbox = ToolboxClient("http://127.0.0.1:5000")
203-
tools = await toolbox.load_toolset()
204-
wrapped_tools = [StructuredTool.from_function(tool, parse_docstring=True) for tool in tools]
205-
model_with_tools = ChatVertexAI(model="gemini-2.0-flash-001").bind_tools(wrapped_tools)
236+
async with ToolboxClient("http://127.0.0.1:5000") as toolbox:
237+
tools = await toolbox.load_toolset()
238+
wrapped_tools = [StructuredTool.from_function(tool, parse_docstring=True) for tool in tools]
239+
model_with_tools = ChatVertexAI(model="gemini-2.0-flash-001").bind_tools(wrapped_tools)
206240

207-
def call_model(state: MessagesState):
208-
messages = state["messages"]
209-
response = model_with_tools.invoke(messages)
210-
return {"messages": [response]}
241+
def call_model(state: MessagesState):
242+
messages = state["messages"]
243+
response = model_with_tools.invoke(messages)
244+
return {"messages": [response]}
211245

212-
def should_continue(state: MessagesState):
213-
messages = state["messages"]
214-
last_message = messages[-1]
215-
if last_message.tool_calls:
216-
return "tools"
217-
return END
246+
def should_continue(state: MessagesState):
247+
messages = state["messages"]
248+
last_message = messages[-1]
249+
if last_message.tool_calls:
250+
return "tools"
251+
return END
218252

219-
workflow = StateGraph(MessagesState)
253+
workflow = StateGraph(MessagesState)
220254

221-
workflow.add_node("agent", call_model)
222-
workflow.add_node("tools", ToolNode(wrapped_tools))
255+
workflow.add_node("agent", call_model)
256+
workflow.add_node("tools", ToolNode(wrapped_tools))
223257

224-
workflow.add_edge(START, "agent")
225-
workflow.add_conditional_edges("agent", should_continue, ["tools", END])
226-
workflow.add_edge("tools", "agent")
258+
workflow.add_edge(START, "agent")
259+
workflow.add_conditional_edges("agent", should_continue, ["tools", END])
260+
workflow.add_edge("tools", "agent")
227261

228-
app = workflow.compile()
262+
app = workflow.compile()
229263
```
230264

231265
## Client to Server Authentication
@@ -264,16 +298,16 @@ You can configure these dynamic headers in two ways:
264298
```python
265299
from toolbox_core import ToolboxClient
266300

267-
client = ToolboxClient("toolbox-url", headers={"header1": header1_getter, "header2": header2_getter, ...})
301+
async with ToolboxClient("toolbox-url", headers={"header1": header1_getter, "header2": header2_getter, ...}) as client:
268302
```
269303

270304
1. **After Client Initialization**
271305

272306
```python
273307
from toolbox_core import ToolboxClient
274308

275-
client = ToolboxClient("toolbox-url")
276-
client.add_headers({"header1": header1_getter, "header2": header2_getter, ...})
309+
async with ToolboxClient("toolbox-url") as client:
310+
client.add_headers({"header1": header1_getter, "header2": header2_getter, ...})
277311
```
278312

279313
### Authenticating with Google Cloud Servers
@@ -299,13 +333,13 @@ For Toolbox servers hosted on Google Cloud (e.g., Cloud Run) and requiring
299333
from toolbox_core import auth_methods
300334

301335
auth_token_provider = auth_methods.aget_google_id_token # can also use sync method
302-
client = ToolboxClient(
336+
async with ToolboxClient(
303337
URL,
304338
client_headers={"Authorization": auth_token_provider},
305-
)
306-
tools = await client.load_toolset()
339+
) as client:
340+
tools = await client.load_toolset()
307341

308-
# Now, you can use the client as usual.
342+
# Now, you can use the client as usual.
309343
```
310344

311345
## Authenticating Tools
@@ -378,17 +412,17 @@ You can add the token retriever function to a tool object *after* it has been
378412
loaded. This modifies the specific tool instance.
379413

380414
```py
381-
toolbox = ToolboxClient("http://127.0.0.1:5000")
382-
tool = await toolbox.load_tool("my-tool")
415+
async with ToolboxClient("http://127.0.0.1:5000") as toolbox:
416+
tool = await toolbox.load_tool("my-tool")
383417

384-
auth_tool = tool.add_auth_token_getter("my_auth", get_auth_token) # Single token
418+
auth_tool = tool.add_auth_token_getter("my_auth", get_auth_token) # Single token
385419

386-
# OR
420+
# OR
387421

388-
multi_auth_tool = tool.add_auth_token_getters({
389-
"my_auth_1", get_auth_token_1,
390-
"my_auth_2", get_auth_token_2,
391-
}) # Multiple tokens
422+
multi_auth_tool = tool.add_auth_token_getters({
423+
"my_auth_1": get_auth_token_1,
424+
"my_auth_2": get_auth_token_2,
425+
}) # Multiple tokens
392426
```
393427

394428
#### Option B: Add Authentication While Loading Tools
@@ -421,12 +455,12 @@ async def get_auth_token():
421455
# This example just returns a placeholder. Replace with your actual token retrieval.
422456
return "YOUR_ID_TOKEN" # Placeholder
423457

424-
toolbox = ToolboxClient("http://127.0.0.1:5000")
425-
tool = await toolbox.load_tool("my-tool")
458+
async with ToolboxClient("http://127.0.0.1:5000") as toolbox:
459+
tool = await toolbox.load_tool("my-tool")
426460

427-
auth_tool = tool.add_auth_token_getters({"my_auth": get_auth_token})
428-
result = auth_tool(input="some input")
429-
print(result)
461+
auth_tool = tool.add_auth_token_getters({"my_auth": get_auth_token})
462+
result = auth_tool(input="some input")
463+
print(result)
430464
```
431465

432466
## Binding Parameter Values
@@ -456,14 +490,14 @@ Bind values to a tool object *after* it has been loaded. This modifies the
456490
specific tool instance.
457491

458492
```py
459-
toolbox = ToolboxClient("http://127.0.0.1:5000")
460-
tool = await toolbox.load_tool("my-tool")
493+
async with ToolboxClient("http://127.0.0.1:5000") as toolbox:
494+
tool = await toolbox.load_tool("my-tool")
461495

462-
bound_tool = tool.bind_param("param", "value")
496+
bound_tool = tool.bind_param("param", "value")
463497

464-
# OR
498+
# OR
465499

466-
bound_tool = tool.bind_params({"param": "value"})
500+
bound_tool = tool.bind_params({"param": "value"})
467501
```
468502

469503
### Option B: Binding Parameters While Loading Tools
@@ -490,8 +524,8 @@ invoked to dynamically determine the parameter's value at runtime.
490524

491525
```py
492526
async def get_dynamic_value():
493-
# Logic to determine the value
494-
return "dynamic_value"
527+
# Logic to determine the value
528+
return "dynamic_value"
495529

496530
dynamic_bound_tool = tool.bind_param("param", get_dynamic_value)
497531
```

0 commit comments

Comments
 (0)