feat: integrate ollama api with palette by klown · Pull Request #42 · inclusive-design/adaptive-palette

klown · 2026-01-12T19:50:38Z

This pull request has been tested by running npm test without errors
This pull request has been built by running npm run dev without errors
This pull request has been built by running npm run serveAppsDemos without errors
This isn't a duplicate of an existing pull request

Description

This adds the ability for the adaptive-palette to interace with local LLMs using the ollama api.

At present, the query is limited to generate suggestions for full sentences. Also, since there is no UI to choose among LLMs, the LLM is "fixed" as llama3.1-8B.

There is a dialog and supporting software that permits users to edit and save the query they wish to send to llama3.1. These prompts are saved in the browser's local storage.

Steps to test

To query the LLM:

Add symbols to the input area using the various available palettes, including the indicator and modifier palettes,
Click the "Telegraphic completions button",
Wait for the top five suggested full sentences,
Click one of the suggestions to hear it spoken, or
Click "Cancel" to remove all suggestions.

To modify or add prompts:

Click the "Prompt DIalog" expansion button,
Choose a prompt from the select menu,
Modify the text in the "Prompt:" textarea, or completely change it,
Use the "Save above prompt as:" button and its accompanying text input field to save the prompt with a new name or the same name to overwrite an existing prompt.

Additional information

The UI is very minimal and invites improvement. The main point of this PR is to show how ollama can be integrated with the adaptive-palette.

…arate file

Also, add `stream` (`true` or `false`) parameter to `queryChat()` function.

…pi-with-palette

- `CommandTeletgraphicCompletions` is component that calls ollama with a "bag of words" and a prompt to provide sentence completions - Introduced `sentenceCompletionsSignal` that will eventually handle changes to the completions.

…entry

…esting

- removed all `console.debug()` statements

…pi-with-palette

- ISAAC demo branch: feat/find-by-gloss-p[us-modifiers

- ISAAC demo branch: feat/find-by-gloss-plus-modifiers

cindyli · 2026-01-12T19:56:01Z

apps/ollama/ollama.js

@@ -1,5 +1,5 @@
 /*
- * Copyright 2024 Inclusive Design Research Centre, OCAD University
+ * Copyright 2024-2025 Inclusive Design Research Centre, OCAD University


The copyright can be up to 2026 now. :)

cindyli · 2026-01-13T23:53:06Z

src/client/ActionTextCell.ts

+  const gridStyles = generateGridStyle(columnStart, columnSpan, rowStart, rowSpan);
+
+  const sentenceClicked = () => {
+    speak(label.replace(/^[0-9]+./,"").trim());  // remove any leading "1. "


Probably improve the code comment to: Remove a leading list number, e.g., "1. "

Okay, comment improved.

cindyli · 2026-01-13T23:53:34Z

src/client/ActionTextCell.ts

+ * https://github.com/inclusive-design/adaptive-palette/blob/main/LICENSE
+ */
+
+import { VNode } from "preact";


There isn't a test for this component. Is it intentional?

No there should be a test. It's possible I misplaced it. If not, I will write one.

cindyli · 2026-01-14T01:07:57Z

src/client/ollamaApi.ts

+  if (streamResp) {
+    return await ollama.chat({ ...request, stream: true });
+  }
+  else {
+    return await ollama.chat({ ...request, stream: false });
+  }


Since streamResp is already a boolean, would it better to use it as the stream value:

return await ollama.chat({ ...request, stream: streamResp });

That way of writing it doesn't work when running the tests. It fails with an error message that ends, Type 'boolean' is not assignable to type 'true', or Type 'boolean' is not assignable to type 'false'.

I wrote a lengthy comment at the beginning of the function, which shows the full error message, and explains the use of the if/else statement. The problem with your suggestion is that streamResp is of type boolean, which does not match either type true or false. I know that it should work, but it doesn't. How true and false can be interpreted as types is beyond me ...

@klown, Gemini suggested this fix and it works:

return await ollama.chat({ ...request, stream: streamResp } as any);

Its explanation:
This error occurs because the Ollama library uses TypeScript function overloads to change the return type based on the value of stream.

The ollama.chat method is defined roughly like this:

Overload 1: If stream is exactly true, it returns an AsyncIterator.

Overload 2: If stream is exactly false (or missing), it returns a standard Promise.

When you pass a variable streamResp of type boolean, TypeScript doesn't know at compile-time whether it will be true or false. Since boolean is a union of true | false, it fails to match either specific overload.

Using as any here is generally safe because you are just telling the compiler to stop over-analyzing the overload.

It's Gemini vs. StackOverflow!l That is, I found a StackOverflow article that recommends the same solution that I used.

Gemini's explanation is what I already understood, except for the use of any as the return type -- that's new, and does not match anything else I have found. I guess Gemini's training data is more extensive than my search.

While the solution works, it introduces a lint error. The linter objects to the use of any. As a compromise, I tried to use the actual return types instead of any:

return await ollama.chat({ ...request, stream: streamResp } as Promise<AbortableAsyncIterator<ChatResponse>> | Promise<ChatResponse>);

While it solves the lint error, and the code runs, the tests fail with an error about the two return types not overlapping, and a suggestion to use unknown:

return await ollama.chat({ ...request, stream: streamResp } as unknown);

But that didn't work either. So, it's a question of putting in another lint exception or leaving the code as is.

Footnote: does Gemini give any reason why this is needed only when running tests? As I noted, the error was non-existent when just running the code.

You are right, this approach fails the linting check. I looked at other Gemini-suggested solutions. They all look like overkills. Seems better to stay with the if/else. :)

Gemini's explanation regarding why it works with vite but not jest:

Vite: By default, Vite uses esbuild to transpile TypeScript. Esbuild is a "transpile-only" tool—it strips types and converts code to JavaScript without performing full type checking. Unless you are running a separate tsc --noEmit command, Vite will ignore this error.

Jest (ts-jest): When running tests, ts-jest often performs a full type check or uses a stricter configuration. It identifies that boolean is not specific enough to satisfy either true or false literals in the overloaded signature.

cindyli · 2026-01-14T22:33:53Z

src/client/SentenceCompletionsPalette.ts

+ * You may obtain a copy of the License at
+ * https://github.com/inclusive-design/adaptive-palette/blob/main/LICENSE
+ */
+import { html } from "htm/preact";


There isn't a test for this palette which is OK if this is a temporary UI solution.

I went ahead and wrote a test. As a side effect, it revealed a few minor issues that I have since fixed. So, even if it is temporary, at least it's well described.

cindyli · 2026-01-14T23:07:53Z

src/client/DialogPromptEntries.ts

+  // determine the <select>'s selected index.
+  const options = [];
+  const localStore = window.localStorage;
+  for (let i = 0; i < localStore.length; i++) {


It would be better to read localStorage only once when the component mounts. Can this be achieved by using useEffect?

I think this has been overtaken by other modifications. The use of useState() to handle the prompts display has as a side effect removed this from the code.

cindyli · 2026-01-14T23:21:52Z

src/client/DialogPromptEntries.ts

+  // `selectSelectedIndex` or (2) from the <select>'s current `selectedIndex`
+  // property.
+  let thePrompt = "";
+  const theSelect = document.getElementById("promptSelect") as HTMLSelectElement;


It doesn't feel right using document.getElementById() to read and update directly in the real DOM. Can you use useState to track and update values.

The same comment applies to the read and update of promptName and systemPrompt below. I wonder if you could use onInput listeners to do the update when these values change.

Thanks to your and Gemini's help this has been done.

cindyli · 2026-01-14T23:34:20Z

src/client/CommandTelegraphicCompletions.ts

+  model: string,
+  // This should be a boolean type, but for Preact/html the type of the value is
+  // string.  It should be "true" or "false".
+  stream: ("true" | "false")


It seems Preact has native boolean support by directly using stream: boolean, and the use of streamAsBoolean can be removed.

Thanks for making me revisit this @cindyli. While I couldn't solve it back in the spring, I finally found the solution.

While it was always possible to declare the "stream" property as "boolean" (stream: boolean), it doesn't matter in some sense. If the render call is done like so:

render( html`<${CommandTelegraphicCompletions} model="llama3.1:latest" stream=false />`, ...

Then typeof props.stream returns 'string', not 'boolean', even though props.stream is declared a boolean. That was mystifying to me, frankly. However, the correct way to render the component is:

render( html`<${CommandTelegraphicCompletions} model="llama3.1:latest" stream=${false} />`, ...

In that case typeof props.stream is 'boolean', and the conditional to change the string to an actual boolean using streamAsBoolean can be removed.

Smart fix. Thanks for sharing. Now I know. :)

Yeah, there are so many little fiddly gotchas with Preact, html, TypeScript, and jest...

cindyli · 2026-01-14T23:39:16Z

src/client/CommandTelegraphicCompletions.ts

+    changeEncodingContents.value.payloads.forEach( (value) => {
+      labelText.push(value.label);
+    });
+    const systemPrompt = (document.getElementById(TEXTAREA_ID) as HTMLTextAreaElement).value;


Same concern about using document.getElementById to reach into the real DOM rather than using Preact way.

In this case, I don't see what the Preact way is, exactly. The systemPrompt ig in another part of the DOM, or, from Preact's perspective, in another component, the DialogPromptEntries component. I thought of using useState(), but that is specific to the current CommandTelegraphicCompletions component. Another possiblity is to useContext(), but that seems to be for (re)rendering components, not handling the user clicking the "Telegraphic Completions" button. Perhaps using another signal is the appropriate route, but again, the user changing the prompt in the DialogPromptEntries component should not cause the CommandTelegraphicCompletions component to do anything, neither re-render nor ask for completions. It's only when the user asks for completions that the CommandTelegraphicCompletions do anything.

I'm leaving it as it is for now, but still thinking about it.

cindyli · 2026-01-14T23:41:03Z

src/client/CommandTelegraphicCompletions.ts

+      <button onClick=${getTelegraphicCompletions}>
+        ${TELEGRPAHIC_BUTTON_LABEL}
+      </button>
+      <span style="visibility: hidden">F</span>


Why is there a text "F" in the <span>? If it's for spacing, is it possible to use margin?

Actually, it was for debugging. But, removing it makes the buttons too close. Using margin ins the scss file is a good idea -- done.

cindyli · 2026-01-14T23:51:16Z

The other issue we talked earlier is the color of the prompt palette looks different in my computer:

When calling `render(...)`, the correct way to pass booleans is by using ${true} or ${false}, e.g.: `render(html`<$ Component stream=${false} ...);`

… string

cindyli · 2026-01-21T23:24:29Z

The difference in background and foreground colors for buttons and input fields in Firefox is caused by my OS being set to a dark color scheme. Firefox’s color scheme setting is set to Automatic, so it inherits the system preference.

If you change your browser’s color scheme to Dark, you should be able to reproduce this issue.

Our CSS should accommodate system supported color schemes. If foreground and background colors are not explicitly set, they should automatically adapt to the selected color scheme, which should fix this problem.

cindyli · 2026-01-23T23:26:11Z

getSvgMarkupString() in the SvgUtils.ts doesn't seem to be used anywhere except in its test.

cindyli · 2026-01-26T14:53:27Z

src/client/index.js


+render(html`<${DialogPromptEntries} />`, document.getElementById("llm_prompt"));
+render(
+  html`<${CommandTelegraphicCompletions} model="llama3.1:latest" stream=${false} />`,


Hardcoding the model name to llama3.1:latest is problematic. On my computer, the same model is installed as llama3.1:8b when downloaded from Ollama, so the app cannot find it and the UI gets stuck at “Working…”.

It's probably better to do what you used to do:

Dynamically list available Ollama models in a dropdown and let users select one.

Detect and report errors when Ollama is not installed or when no models are available, and disable buttons and the text area.

Or, check with the design team to clarify the expected workflow.

Agreed, I have added another <select> element to allow users to choose among installed models and update the state of the buttons as appropriate.

cindyli · 2026-01-26T15:02:45Z

Another issue I noticed is that when the input sentence is empty or null, clicking “Telegraphic Completions” still sends an empty string to the model, and came back with a response like: I'm afraid you forgot to provide the telegraphic speech….

It would be better to add a client-side guard to validate that an input sentence exists before making the request, which would avoid unnecessary backend round trips.

…nce completions palette

- compatible to browser's light/dark mode switch, - spacing (margin) between the two buttons.

klown · 2026-03-02T20:53:05Z

Another issue I noticed is that when the input sentence is empty or null, clicking “Telegraphic Completions” still sends an empty string to the model, and came back with a response like: I'm afraid you forgot to provide the telegraphic speech….

It would be better to add a client-side guard to validate that an input sentence exists before making the request, which would avoid unnecessary backend round trips.

The changes I made when I added the dropdown for selecting a language model includes disabling the "Telegraphic Completions" and "Cancel" buttons (part of the CommandTelegraphicCompletions component) where there is no content in the symbol input area. That was done by using the existing changeEncodingContents signal. So, the user can not longer pass an empty sentence for completion.

Also regarding the dark-mode issue, as noted in PR #44, dark-mode is disabled for now, and there is a new issue #51 that requires work to fix it. As I noted in PR #44, it should be merged first and then this one merged.

So: ready for another round of reviews, thanks.

klown added 28 commits March 28, 2025 11:06

feat: refactor ollama app to separate out ollama api calls into a sep…

b836ae5

…arate file

feat: move ollamaApi.ts to src/client for use by adpative-palette

8bf9fb4

feat: add types for the typescript version of the ollama api

2fc8ec1

Also, add `stream` (`true` or `false`) parameter to `queryChat()` function.

Merge branch 'feat/standard-bliss-board' into feat/integrate-ollama-a…

4369d09

…pi-with-palette

feat: add dynamic sentence completions palette

c1e2ddc

feat: actually commit the SentenceCompletionsPalette component

63058c7

feat: add dialog for changing the system prompt for querying ollama

aa726b8

feat: improve dialog for system prompt and add dialog for svg string …

2faf0f0

…entry

fix: insure that stream constant is actually a boolean

94fb28f

feat: properly update the prompt dialog when a new prompt is added

a30918c

feat: move styles of dialogs to their own .scss files

fb33aa3

feat: add some unit tests for the new prompt dialog

b670284

chore: use constants for form control ids and values

14227d4

fix: indent typo

fc50c27

feat: add tests for ActionSvgEntryField component

ef019df

feat: add a way to follow the typing restrictions of ollama.ts when t…

1e6ac7f

…esting

feat: add tests for ollama api

64be8a7

- removed all `console.debug()` statements

Merge branch 'feat/standard-bliss-board' into feat/integrate-ollama-a…

e0b7d2d

…pi-with-palette

Merge branch 'main' into feat/integrate-ollama-api-with-palette

07d5318

Merge branch 'main' into feat/integrate-ollama-api-with-palette

4e7d1d0

Merge branch 'main' into feat/integrate-ollama-api-with-palette

751372d

fix: improve contrast of text on sentence completion buttons

ac981d7

Merge branch 'main' into feat/integrate-ollama-api-with-palette

c44ebc6

fix: add missing ","

1e10fe5

fix: remove svg entry UI and related code, and duplicate go-back button

c621a03

chore: minor fixes to align with ISAAC demo branch

8df804e

- ISAAC demo branch: feat/find-by-gloss-p[us-modifiers

feat: copy main display differences from ISAAC demo branch

f0b8d93

- ISAAC demo branch: feat/find-by-gloss-plus-modifiers

klown requested a review from cindyli January 12, 2026 19:51

cindyli requested changes Jan 14, 2026

View reviewed changes

klown added 4 commits January 20, 2026 11:01

fix; passing a boolean value to a preact component using ${...}

63427b1

When calling `render(...)`, the correct way to pass booleans is by using ${true} or ${false}, e.g.: `render(html`<$ Component stream=${false} ...);`

fix: comments in the code

aab3678

feat: added tests for ActionTextCell and SentenceCompletionsPalette

14aee33

fix: typo where "R" was errorneously used for "$" in an html template…

bfd5478

… string

cindyli mentioned this pull request Jan 23, 2026

feat: find symbols by searching the gloss, and enter symbols using svg builder strings #44

Open

3 tasks

cindyli reviewed Jan 26, 2026

View reviewed changes

klown added 3 commits January 28, 2026 11:42

feat: use states for tracking prompt dialog ui

2a4b6c6

fix: remove unnecessary call to the palette store to remove the sente…

b5a59be

…nce completions palette

fix: styles of the telegraphic completions buttons

76ed06e

- compatible to browser's light/dark mode switch, - spacing (margin) between the two buttons.

Conversation

klown commented Jan 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Steps to test

To query the LLM:

To modify or add prompts:

Additional information

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cindyli commented Jan 14, 2026

Uh oh!

cindyli commented Jan 21, 2026

Uh oh!

cindyli commented Jan 23, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cindyli commented Jan 26, 2026

Uh oh!

klown commented Mar 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

klown commented Jan 12, 2026 •

edited

Loading