Support bypass tools option on lightspeed-stack /streaming_query API #1737

TamiTakamiya · 2025-08-08T18:47:15Z

Jira Issue: https://issues.redhat.com/browse/AAP-51359

Assisted-by: n/a
Generated by: n/a

Description

Support the "no_tools" option on lightspeed-stack's /streaming_query and /query APIs.

This option is available when the chatbot UI is running in the DEBUG mode. The new "Bypass tools" checkbox
is added on the UI for setting System Prompt Override.

Testing

Steps to test

Pull down the PR
Run Unit tests

Scenarios tested

Unit tests + manual tests using local server

Production deployment

This code change is ready for production on its own
This code change requires the following considerations before going to production:

github-actions · 2025-08-10T18:41:58Z

# npm audit report

@eslint/plugin-kit  <0.3.4
@eslint/plugin-kit is vulnerable to Regular Expression Denial of Service attacks through ConfigCommentParser - https://github.com/advisories/GHSA-xffm-g5w8-qvg7
fix available via `npm audit fix`
node_modules/@eslint/plugin-kit

form-data  4.0.0 - 4.0.3
Severity: critical
form-data uses unsafe random function in form-data for choosing boundary - https://github.com/advisories/GHSA-fjxv-7rqg-78g4
fix available via `npm audit fix`
node_modules/form-data

prismjs  <1.30.0
Severity: moderate
PrismJS DOM Clobbering vulnerability - https://github.com/advisories/GHSA-x7hr-w5r2-h6wg
fix available via `npm audit fix`
node_modules/refractor/node_modules/prismjs
  refractor  <=4.6.0
  Depends on vulnerable versions of prismjs
  node_modules/refractor
    react-syntax-highlighter  >=6.0.0
    Depends on vulnerable versions of refractor
    node_modules/react-syntax-highlighter
      @patternfly/chatbot  <=6.3.0
      Depends on vulnerable versions of react-syntax-highlighter
      node_modules/@patternfly/chatbot

6 vulnerabilities (1 low, 4 moderate, 1 critical)

To address all issues, run:
  npm audit fix

github-actions · 2025-08-10T19:13:28Z

# npm audit report

@eslint/plugin-kit  <0.3.4
@eslint/plugin-kit is vulnerable to Regular Expression Denial of Service attacks through ConfigCommentParser - https://github.com/advisories/GHSA-xffm-g5w8-qvg7
fix available via `npm audit fix`
node_modules/@eslint/plugin-kit

form-data  4.0.0 - 4.0.3
Severity: critical
form-data uses unsafe random function in form-data for choosing boundary - https://github.com/advisories/GHSA-fjxv-7rqg-78g4
fix available via `npm audit fix`
node_modules/form-data

prismjs  <1.30.0
Severity: moderate
PrismJS DOM Clobbering vulnerability - https://github.com/advisories/GHSA-x7hr-w5r2-h6wg
fix available via `npm audit fix`
node_modules/refractor/node_modules/prismjs
  refractor  <=4.6.0
  Depends on vulnerable versions of prismjs
  node_modules/refractor
    react-syntax-highlighter  >=6.0.0
    Depends on vulnerable versions of refractor
    node_modules/react-syntax-highlighter
      @patternfly/chatbot  <=6.3.0
      Depends on vulnerable versions of react-syntax-highlighter
      node_modules/@patternfly/chatbot

6 vulnerabilities (1 low, 4 moderate, 1 critical)

To address all issues, run:
  npm audit fix

github-actions · 2025-08-10T19:20:26Z

# npm audit report

@eslint/plugin-kit  <0.3.4
@eslint/plugin-kit is vulnerable to Regular Expression Denial of Service attacks through ConfigCommentParser - https://github.com/advisories/GHSA-xffm-g5w8-qvg7
fix available via `npm audit fix`
node_modules/@eslint/plugin-kit

form-data  4.0.0 - 4.0.3
Severity: critical
form-data uses unsafe random function in form-data for choosing boundary - https://github.com/advisories/GHSA-fjxv-7rqg-78g4
fix available via `npm audit fix`
node_modules/form-data

prismjs  <1.30.0
Severity: moderate
PrismJS DOM Clobbering vulnerability - https://github.com/advisories/GHSA-x7hr-w5r2-h6wg
fix available via `npm audit fix`
node_modules/refractor/node_modules/prismjs
  refractor  <=4.6.0
  Depends on vulnerable versions of prismjs
  node_modules/refractor
    react-syntax-highlighter  >=6.0.0
    Depends on vulnerable versions of refractor
    node_modules/react-syntax-highlighter
      @patternfly/chatbot  <=6.3.0
      Depends on vulnerable versions of react-syntax-highlighter
      node_modules/@patternfly/chatbot

6 vulnerabilities (1 low, 4 moderate, 1 critical)

To address all issues, run:
  npm audit fix

github-actions · 2025-08-10T19:43:47Z

# npm audit report

@eslint/plugin-kit  <0.3.4
@eslint/plugin-kit is vulnerable to Regular Expression Denial of Service attacks through ConfigCommentParser - https://github.com/advisories/GHSA-xffm-g5w8-qvg7
fix available via `npm audit fix`
node_modules/@eslint/plugin-kit

form-data  4.0.0 - 4.0.3
Severity: critical
form-data uses unsafe random function in form-data for choosing boundary - https://github.com/advisories/GHSA-fjxv-7rqg-78g4
fix available via `npm audit fix`
node_modules/form-data

prismjs  <1.30.0
Severity: moderate
PrismJS DOM Clobbering vulnerability - https://github.com/advisories/GHSA-x7hr-w5r2-h6wg
fix available via `npm audit fix`
node_modules/refractor/node_modules/prismjs
  refractor  <=4.6.0
  Depends on vulnerable versions of prismjs
  node_modules/refractor
    react-syntax-highlighter  >=6.0.0
    Depends on vulnerable versions of refractor
    node_modules/react-syntax-highlighter
      @patternfly/chatbot  <=6.3.0
      Depends on vulnerable versions of react-syntax-highlighter
      node_modules/@patternfly/chatbot

6 vulnerabilities (1 low, 4 moderate, 1 critical)

To address all issues, run:
  npm audit fix

manstis

LGTM 👍

manstis · 2025-08-11T15:27:28Z

aap_chatbot/src/useChatbot/useChatbot.ts

@@ -33,7 +33,9 @@ const botName =
  ANSIBLE_LIGHTSPEED_PRODUCT_NAME;

 export const modelsSupported: LLMModel[] = [
-  { model: "granite-3.3-8b-instruct", provider: "rhoai" },
+  { model: "granite-3.3-8b-instruct", provider: "my_rhoai_dev" },
+  { model: "gemini/gemini-2.5-flash", provider: "gemini" },


Whilst these other models will work with presumably your AI Installer backend they will break LSIA. Are you planning on updating the SaaS backend to support these too?

Ok. I will revert the line 36.

I thought it needed to be consistent with our (road-core based) chatbot configuration (this line), and that's the reason that I changed it to my_rhoai_dev, but the operator (this line) assumes rhoai as the provider name for granite-3.3-8b-instruct, I think I need to modify https://github.com/ansible/ansible-wisdom-ops/pull/1483 to change the provider name.

(additional update) ... Though It is possible to have use provider names in /aap_chatbot and /ansible_ai_connect_chatbot, I want make these code as common as possible...

@TamiTakamiya Operator deployments are different to what we have in ansible-wisdom--ops. It was the Gemini models that'd cause problems as the SaaS backend only supports Granite in llama-stacks run configuration.

Ok. Then I will revert line 36 & 37 of useChatbot.ts under /aap_chatbot, but not the one under /ansible_ai_connect_chatbot. Though they are not displayed in non-debug mode, on-prem chatbot does not support Gemini and it does not require the change and the provider for Granite needs to be remain as rhoai for on-prem version.

sonarqubecloud · 2025-08-13T12:19:28Z

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
97.6% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

romartin

LGTM. Thanks @TamiTakamiya

TamiTakamiya force-pushed the TamiTakamiya/support-chatbot-no-tools-option branch 2 times, most recently from b93fe70 to dae0254 Compare August 8, 2025 19:58

TamiTakamiya force-pushed the TamiTakamiya/support-chatbot-no-tools-option branch from 7b6d07b to ae2dd38 Compare August 10, 2025 19:26

Support bypass tools option on lightspeed-stack /streaming_query API

3766792

TamiTakamiya force-pushed the TamiTakamiya/support-chatbot-no-tools-option branch from ae2dd38 to 3766792 Compare August 10, 2025 19:43

npm audit fix

923a649

TamiTakamiya marked this pull request as ready for review August 10, 2025 23:48

TamiTakamiya requested review from ldjebran, mabashian, romartin and goneri August 10, 2025 23:48

ansible deleted a comment from github-actions bot Aug 10, 2025

Added a comment to a test case

7277258

manstis previously approved these changes Aug 11, 2025

View reviewed changes

TamiTakamiya dismissed manstis’s stale review via bb98a1e August 11, 2025 17:42

Revert provider name for Granite used in on-prem mode

8516186

TamiTakamiya force-pushed the TamiTakamiya/support-chatbot-no-tools-option branch from bb98a1e to 8516186 Compare August 13, 2025 11:49

TamiTakamiya added 2 commits August 13, 2025 08:00

Hide gemini-2.5-pro for now

7feb311

Remove a comment line

06b30f1

romartin approved these changes Aug 13, 2025

View reviewed changes

TamiTakamiya merged commit 76504a7 into main Aug 13, 2025
14 checks passed

TamiTakamiya deleted the TamiTakamiya/support-chatbot-no-tools-option branch August 13, 2025 13:20

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Support bypass tools option on lightspeed-stack /streaming_query API #1737

Support bypass tools option on lightspeed-stack /streaming_query API #1737

Uh oh!

TamiTakamiya commented Aug 8, 2025

Uh oh!

github-actions bot commented Aug 10, 2025

Uh oh!

github-actions bot commented Aug 10, 2025

Uh oh!

github-actions bot commented Aug 10, 2025

Uh oh!

github-actions bot commented Aug 10, 2025

Uh oh!

manstis left a comment

Uh oh!

manstis Aug 11, 2025

Uh oh!

TamiTakamiya Aug 11, 2025

Uh oh!

TamiTakamiya Aug 11, 2025

Uh oh!

manstis Aug 11, 2025

Uh oh!

TamiTakamiya Aug 13, 2025

Uh oh!

sonarqubecloud bot commented Aug 13, 2025

Uh oh!

romartin left a comment

Uh oh!

Uh oh!

Uh oh!

Support bypass tools option on lightspeed-stack /streaming_query API #1737

Support bypass tools option on lightspeed-stack /streaming_query API #1737

Uh oh!

Conversation

TamiTakamiya commented Aug 8, 2025

Description

Testing

Steps to test

Scenarios tested

Production deployment

Uh oh!

github-actions bot commented Aug 10, 2025

Uh oh!

github-actions bot commented Aug 10, 2025

Uh oh!

github-actions bot commented Aug 10, 2025

Uh oh!

github-actions bot commented Aug 10, 2025

Uh oh!

manstis left a comment

Choose a reason for hiding this comment

Uh oh!

manstis Aug 11, 2025

Choose a reason for hiding this comment

Uh oh!

TamiTakamiya Aug 11, 2025

Choose a reason for hiding this comment

Uh oh!

TamiTakamiya Aug 11, 2025

Choose a reason for hiding this comment

Uh oh!

manstis Aug 11, 2025

Choose a reason for hiding this comment

Uh oh!

TamiTakamiya Aug 13, 2025

Choose a reason for hiding this comment

Uh oh!

sonarqubecloud bot commented Aug 13, 2025

Quality Gate passed

Uh oh!

romartin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!