Skip to content

feat: add support for USE_STRUCTURED_RESPONSE (v2)#799

Merged
suwhang-cisco merged 23 commits intomainfrom
feat/add-structured-response-v2
Feb 13, 2026
Merged

feat: add support for USE_STRUCTURED_RESPONSE (v2)#799
suwhang-cisco merged 23 commits intomainfrom
feat/add-structured-response-v2

Conversation

@suwhang-cisco
Copy link
Collaborator

Description

  • Add a new support for USE_STRUCTURED_RESPONSE where LLM uses structured response tool call as its final output.
  • This flag is false by default (agent instructed to use [Final Answer] stamps instead.)
  • When enabled, agent will always do a tool call at the end for the final output, thus reducing any possibility of duplicate outputs.
  • When enabled, request_user_input tool will be dropped from the agent as the structured response itself has the ability to include this (fields "user_input": True, and "input_fields")
  • Introduce a new field was_task_successful to indicate the task success and instruct the agent to set is_task_complete to true to indicate this is the FINAL output to be shown to the user.

Type of Change

  • Bugfix
  • New Feature
  • Breaking Change
  • Refactor
  • Documentation
  • Other (please describe)

Pre-release Helm Charts (Optional)

For chart changes, you can test pre-release versions before merging:

  • Base repo contributors: Create a branch starting with pre/ for automatic pre-release builds
  • Fork contributors: Ask a maintainer to add the helm-prerelease label
  • Pre-release charts are published to ghcr.io/cnoe-io/pre-release-helm-charts
  • Cleanup happens automatically when the PR closes or label is removed

Checklist

  • I have read the contributing guidelines
  • Existing issues have been referenced (where applicable)
  • I have verified this change is not present in other open pull requests
  • Functionality is documented
  • All code style checks pass
  • New code contribution is covered by automated tests
  • All new and existing tests pass

…ed response tool call as its final output

Signed-off-by: suwhang-cisco <suwhang@cisco.com>
Signed-off-by: suwhang-cisco <suwhang@cisco.com>
…put if USE_STRUCTURED_RESPONSE is true

Signed-off-by: suwhang-cisco <suwhang@cisco.com>
@github-actions
Copy link
Contributor

✅ No proprietary content detected. This PR is clear for review!

1 similar comment
@github-actions
Copy link
Contributor

✅ No proprietary content detected. This PR is clear for review!

@github-actions
Copy link
Contributor

📊 Test Coverage Report

Main Tests Coverage

Metric Coverage Details
Lines 49.1% 4921/10018 lines
Branches 0.0% 0/0 branches

📁 Coverage Artifacts

  • Main tests: coverage-reports-main artifact
  • RAG tests: coverage-reports-rag artifact (not available)
  • Download artifacts to view detailed HTML coverage reports

@github-actions
Copy link
Contributor

📊 Test Coverage Report

Main Tests Coverage

Metric Coverage Details
Lines 49.1% 4921/10018 lines
Branches 0.0% 0/0 branches

📁 Coverage Artifacts

  • Main tests: coverage-reports-main artifact
  • RAG tests: coverage-reports-rag artifact (not available)
  • Download artifacts to view detailed HTML coverage reports

Signed-off-by: suwhang-cisco <suwhang@cisco.com>
@github-actions
Copy link
Contributor

✅ No proprietary content detected. This PR is clear for review!

@github-actions
Copy link
Contributor

📊 Test Coverage Report

Main Tests Coverage

Metric Coverage Details
Lines 49.1% 4921/10018 lines
Branches 0.0% 0/0 branches

📁 Coverage Artifacts

  • Main tests: coverage-reports-main artifact
  • RAG tests: coverage-reports-rag artifact (not available)
  • Download artifacts to view detailed HTML coverage reports

suwhang-cisco and others added 7 commits February 13, 2026 01:01
…ed response tool call as its final output

Signed-off-by: suwhang-cisco <suwhang@cisco.com>
Signed-off-by: suwhang-cisco <suwhang@cisco.com>
…put if USE_STRUCTURED_RESPONSE is true

Signed-off-by: suwhang-cisco <suwhang@cisco.com>
Signed-off-by: suwhang-cisco <suwhang@cisco.com>
…NSE feature

New test files:
- test_response_format.py (32 tests): PlatformEngineerResponse,
  Metadata, InputField Pydantic model validation, serialization,
  roundtrips, defaults, edge cases
- test_prompts.py (18 tests): generate_system_prompt with
  use_structured_response flag, FINAL_ANSWER_MARKER_SECTION
  inclusion/exclusion, agent descriptions, RAG instructions,
  YAML vs fallback template paths
- test_handle_structured_response.py (22 tests): Pydantic object,
  dict, and string inputs; JSON parsing with markdown fences;
  multiple JSON objects; [FINAL ANSWER] marker detection;
  metadata/input_fields propagation; fallback behavior
- test_executor_structured_response.py (22 tests): from_response_format_tool
  handling in execute(), _handle_task_complete content bypass,
  _handle_user_input_required with metadata form fields,
  event ordering priority, StreamState defaults, new_data_artifact

Also fix lint errors:
- Remove unused `re` import from agent.py
- Fix f-string without placeholders in agent.py
- Fix pre-existing lint issues in test_base_langgraph_agent.py
  and test_base_langgraph_agent_executor.py

Signed-off-by: Sri Aradhyula <sraradhy@cisco.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Add platform_engineer/tests/ and utils/a2a_common/tests/ to the
test-multi-agents target so all new unit tests run as part of
make test.

Signed-off-by: Sri Aradhyula <sraradhy@cisco.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@sriaradhyula sriaradhyula force-pushed the feat/add-structured-response-v2 branch from 6e7b367 to c3b832e Compare February 13, 2026 07:02
@github-actions
Copy link
Contributor

✅ No proprietary content detected. This PR is clear for review!

@github-actions
Copy link
Contributor

📊 Test Coverage Report

Main Tests Coverage

Metric Coverage Details
Lines 59.7% 6768/11330 lines
Branches 0.0% 0/0 branches

📁 Coverage Artifacts

  • Main tests: coverage-reports-main artifact
  • RAG tests: coverage-reports-rag artifact (not available)
  • Download artifacts to view detailed HTML coverage reports

…ai-platform-engineering into feat/add-structured-response-v2
Signed-off-by: suwhang-cisco <suwhang@cisco.com>
Signed-off-by: suwhang-cisco <suwhang@cisco.com>
@github-actions
Copy link
Contributor

✅ No proprietary content detected. This PR is clear for review!

1 similar comment
@github-actions
Copy link
Contributor

✅ No proprietary content detected. This PR is clear for review!

…al output using field_values

Signed-off-by: suwhang-cisco <suwhang@cisco.com>
…ai-platform-engineering into feat/add-structured-response-v2
@github-actions
Copy link
Contributor

✅ No proprietary content detected. This PR is clear for review!

@github-actions
Copy link
Contributor

github-actions bot commented Feb 13, 2026

🧪 CAIPE UI Test Results

All tests passed

🔴 Overall Coverage: 30%

Coverage
lines
statements
functions
branches

📊 Detailed Coverage

Metric Covered Total Percentage
Lines 3025 9512 31.80%
Statements 3228 10319 31.28%
Functions 537 2015 26.65%
Branches 1933 6803 28.41%

✅ Test Suites

  • ✅ auth-guard.test.tsx - Route protection & authorization
  • ✅ token-expiry-guard.test.tsx - Token expiry handling
  • ✅ a2a-sdk-client.test.ts - A2A streaming SDK
  • ✅ auth-utils.test.ts - Authentication utilities (100% coverage)
  • ✅ auth-config.test.ts - OIDC configuration
📈 Coverage Thresholds
Threshold Target Current Status
Minimum 40% 30% ❌ Fail
Good 60% 30% ⚠️ Below target
Excellent 80% 30% ⚠️ Below target
⚠️ Areas Needing Tests

High Priority:

  • hooks/use-a2a-streaming.ts - Core streaming functionality
  • store/chat-store.ts - Chat state management
  • store/agent-config-store.ts - Agent configuration
  • lib/api-client.ts - API communication
  • lib/storage-mode.ts - MongoDB/localStorage switching

Medium Priority:

  • components/chat/ChatPanel.tsx - Main chat interface
  • components/agent-builder/* - Agent builder UI
  • lib/mongodb.ts - MongoDB integration

💡 Run locally: make caipe-ui-tests
📦 Full report: Check workflow artifacts

Signed-off-by: suwhang-cisco <suwhang@cisco.com>
@github-actions
Copy link
Contributor

✅ No proprietary content detected. This PR is clear for review!

@github-actions
Copy link
Contributor

📊 Test Coverage Report

Main Tests Coverage

Metric Coverage Details
Lines 61.1% 7172/11734 lines
Branches 0.0% 0/0 branches

📁 Coverage Artifacts

  • Main tests: coverage-reports-main artifact
  • RAG tests: coverage-reports-rag artifact (not available)
  • Download artifacts to view detailed HTML coverage reports

Signed-off-by: suwhang-cisco <suwhang@cisco.com>
@github-actions
Copy link
Contributor

✅ No proprietary content detected. This PR is clear for review!

@github-actions
Copy link
Contributor

📊 Test Coverage Report

Main Tests Coverage

Metric Coverage Details
Lines 61.1% 7172/11732 lines
Branches 0.0% 0/0 branches

📁 Coverage Artifacts

  • Main tests: coverage-reports-main artifact
  • RAG tests: coverage-reports-rag artifact (not available)
  • Download artifacts to view detailed HTML coverage reports

@suwhang-cisco suwhang-cisco merged commit c1ff734 into main Feb 13, 2026
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants