Skip to content

Commit 61c2314

Browse files
committed
SimpleChatTC:Readme: Updated wrt new relativelyProper toolCallsHS
Also update the sliding window context size to last 9 chat messages so that there is a sufficiently large context for multi turn tool calls based adjusting by ai and user, without needing to go full hog, which has the issue of overflowing the currently set context window wrt the loaded ai model.
1 parent a644cb3 commit 61c2314

File tree

2 files changed

+17
-8
lines changed

2 files changed

+17
-8
lines changed

tools/server/public_simplechat/readme.md

Lines changed: 14 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -136,8 +136,9 @@ Once inside
136136
called, then the ai response might include request for tool call.
137137
* the SimpleChat client will show details of the tool call (ie tool name and args passed) requested
138138
and allow the user to trigger it as is or after modifying things as needed.
139+
NOTE: Tool sees the original tool call only, for now
139140
* inturn returned / generated result is placed into user query entry text area with approriate tags
140-
ie <tool_response> generated result </tool_response>
141+
ie <tool_response> generated result with meta data </tool_response>
141142
* if user is ok with the tool response, they can click submit to send the same to the GenAi/LLM.
142143
User can even modify the response generated by the tool, if required, before submitting.
143144

@@ -193,7 +194,7 @@ It is attached to the document object. Some of these can also be updated using t
193194
sent back to the ai model, under user control.
194195

195196
as tool calling will involve a bit of back and forth between ai assistant and end user, it is
196-
recommended to set iRecentUserMsgCnt to 5 or more, so that enough context is retained during
197+
recommended to set iRecentUserMsgCnt to 10 or more, so that enough context is retained during
197198
chatting with ai models with tool support.
198199

199200
apiEP - select between /completions and /chat/completions endpoint provided by the server/ai-model.
@@ -239,7 +240,7 @@ It is attached to the document object. Some of these can also be updated using t
239240
be set if needed using the settings ui.
240241

241242
iRecentUserMsgCnt - a simple minded SlidingWindow to limit context window load at Ai Model end.
242-
This is set to 5 by default. So in addition to latest system message, last/latest iRecentUserMsgCnt
243+
This is set to 10 by default. So in addition to latest system message, last/latest iRecentUserMsgCnt
243244
user messages after the latest system prompt and its responses from the ai model will be sent
244245
to the ai-model, when querying for a new response. Note that if enabled, only user messages after
245246
the latest system message/prompt will be considered.
@@ -325,7 +326,7 @@ work.
325326

326327
ALERT: The simple minded way in which this is implemented, it can be dangerous in the worst case,
327328
Always remember to verify all the tool calls requested and the responses generated manually to
328-
ensure everything is fine, during interaction with ai modles with tools support.
329+
ensure everything is fine, during interaction with ai models with tools support.
329330

330331
#### Builtin Tools
331332

@@ -358,9 +359,11 @@ that should be sent back to the ai model, in your constructed code.
358359
Update the tc_switch to include a object entry for the tool, which inturn includes
359360
* the meta data as well as
360361
* a reference to the handler and also
362+
the handler should take toolCallId, toolName and toolArgs and pass these along to
363+
web worker as needed.
361364
* the result key (was used previously, may use in future, but for now left as is)
362365

363-
#### Mapping tool calls and responses to normal assistant - user chat flow
366+
#### OLD: Mapping tool calls and responses to normal assistant - user chat flow
364367

365368
Instead of maintaining tool_call request and resultant response in logically seperate parallel
366369
channel used for requesting tool_calls by the assistant and the resulstant tool role response,
@@ -375,17 +378,22 @@ NOTE: This flow tested to be ok enough with Gemma-3N-E4B-it-Q8_0 LLM ai model fo
375378
given the way current ai models work, most of them should understand things as needed, but need
376379
to test this with other ai models later.
377380

378-
TODO: Need to think later, whether to continue this simple flow, or atleast use tool role wrt
381+
TODO:OLD: Need to think later, whether to continue this simple flow, or atleast use tool role wrt
379382
the tool call responses or even go further and have the logically seperate tool_calls request
380383
structures also.
381384

385+
DONE: rather both tool_calls structure wrt assistant messages and tool role based tool call
386+
result messages are generated as needed.
387+
388+
382389
#### ToDo
383390

384391
WebFetch and Local web proxy/caching server
385392

386393
Try and trap promises based flows to ensure all generated results or errors if any are caught
387394
before responding back to the ai model.
388395

396+
Trap error responses.
389397

390398
### Debuging the handshake
391399

tools/server/public_simplechat/simplechat.js

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -219,7 +219,7 @@ let gUsageMsg = `
219219
<ul class="ul2">
220220
<li> submit tool response placed into user query textarea</li>
221221
</ul>
222-
<li> Default ContextWindow = [System, Last4 Query+Resp, Cur Query].</li>
222+
<li> Default ContextWindow = [System, Last9 Query+Resp, Cur Query].</li>
223223
<ul class="ul2">
224224
<li> ChatHistInCtxt, MaxTokens, ModelCtxt window to expand</li>
225225
</ul>
@@ -983,14 +983,15 @@ class Me {
983983
this.bCompletionFreshChatAlways = true;
984984
this.bCompletionInsertStandardRolePrefix = false;
985985
this.bTrimGarbage = true;
986-
this.iRecentUserMsgCnt = 5;
986+
this.iRecentUserMsgCnt = 10;
987987
/** @type {Object<string, number>} */
988988
this.sRecentUserMsgCnt = {
989989
"Full": -1,
990990
"Last0": 1,
991991
"Last1": 2,
992992
"Last2": 3,
993993
"Last4": 5,
994+
"Last9": 10,
994995
};
995996
this.apiEP = ApiEP.Type.Chat;
996997
/** @type {Object<string, string>} */

0 commit comments

Comments
 (0)