Skip to content

Conversation

@shkurapa
Copy link

Description

This PR introduces a Human-in-the-Loop (HITL) system for the Lungo auction supervisor, enabling human oversight for high-stakes order decisions using LangGraph's interrupt() capability.

Overview

The HITL system uses two specialized models to determine when and how to involve humans in the decision-making process:

  1. WHEN-TO-TRIGGER Model - Analyzes user queries to detect scenarios requiring human review (budget constraints, missing parameters, market volatility)
  2. WHAT-TO-RESPOND Model - Generates actionable scenarios and recommendations for human selection

Key Features

  • LangGraph Interrupts: Pauses graph execution when human intervention is needed, resuming with the user's decision
  • Dynamic Scenario Generation: Generates context-aware allocation scenarios based on order parameters
  • Pluggable Model Architecture: Abstract base classes allow easy replacement of mock models with real inference
  • Full UI Integration: New sidebar section and interactive scenario selection component

New Files

Backend:

  • agents/supervisors/auction/graph/hitl.py - Model interfaces and mock implementations
  • agents/supervisors/auction/graph/graph_hitl.py - HITL-enabled LangGraph with interrupt nodes
  • agents/supervisors/auction/api_hitl.py - REST endpoints for HITL workflow (/agent/prompt/hitl, /agent/prompt/hitl/resume)

Frontend:

  • frontend/src/hooks/useHITLAPI.ts - React hook for HITL API interactions
  • frontend/src/components/Chat/HITLScenarios.tsx - Scenario display and selection component

Modified Files

  • agents/supervisors/auction/main.py - Include HITL router
  • frontend/src/App.tsx - HITL state management
  • frontend/src/components/Chat/ChatArea.tsx - HITL flow integration
  • frontend/src/components/Sidebar/Sidebar.tsx - New "Human-in-the-Loop" navigation section
  • frontend/src/utils/patternUtils.ts - Added PUBLISH_SUBSCRIBE_HITL pattern

Graph Flow

SUPERVISOR
    ├── INVENTORY_SINGLE_FARM → REFLECTION → END
    ├── INVENTORY_ALL_FARMS → REFLECTION → END
    ├── ORDERS → TRIGGER_EVALUATION
    │              ├── [TRIGGER] → RESPONSE_GENERATION → HUMAN_INTERVENTION
    │              │                                          ↓
    │              │                              PROCESS_HUMAN_DECISION → END
    │              └── [NO_TRIGGER] → ORDERS_TOOLS → REFLECTION → END
    └── GENERAL_INFO → END

Example Usage

Prompt: I need to place our Q2 order. 500 lbs total, budget capped at $2,000. What are our options?

The system:

  1. Detects budget constraint, missing origin, and potential budget exceedance
  2. Generates 4 allocation scenarios (Budget-Optimized, Quality-First, Balanced Diversilected scenario and confirms the order

Testing

Select "Human-in-the-Loop > A2A NATS (HITL)" in the sidebar and try the example prompt above.

Implements a two-model HITL system for intelligent human intervention:

1. WHEN-TO-TRIGGER MODEL (Small LLM)
   - Takes raw user query as input
   - Returns decision (TRIGGER/NO_TRIGGER), confidence score, and reasons
   - Detects: budget constraints, missing origin, market volatility, budget exceeded

2. WHAT-TO-RESPOND MODEL (Small LLM)
   - Takes user query + trigger output
   - Generates allocation scenarios with cost/quality analysis
   - Provides recommendations and rationale

New files:
- hitl.py: Model interfaces, data classes, and mock implementations
- graph_hitl.py: LangGraph with interrupt() support for async human review
- api_hitl.py: REST endpoints for HITL workflow (/agent/prompt/hitl, /resume)

Flow:
1. User query -> WHEN-TO-TRIGGER model -> if confidence >= 65%
2. WHAT-TO-RESPOND model generates scenarios
3. interrupt() pauses execution, returns options to caller
4. Human reviews and selects scenario
5. resume() continues execution with human decision

Mock implementations ready for real model integration via HITLModelFactory.
…onses

- Strip trailing whitespace from farm inventory responses to avoid
  Bedrock Claude error: 'final assistant content cannot end with trailing whitespace'
- Applied fix to both single farm and all farms inventory nodes
- Add HITLScenarios component for displaying HITL options to user
- Add useHITLAPI hook for HITL API interactions
- Update ChatArea to integrate HITL flow and scenario selection
- Update App.tsx with HITL state management and callbacks
- Add PUBLISH_SUBSCRIBE_HITL pattern to patternUtils
- Add Human-in-the-Loop section to Sidebar navigation
Based on code review standards:

Documentation:
- Enhanced file headers with purpose, architecture, and example usage
- Added comprehensive docstrings with Args/Returns sections
- Added inline comments explaining complex logic

Type Hints:
- Added return type hints (Dict[str, Any]) to node methods
- Added Optional type hints for nullable variables

Exception Handling:
- Changed logger.error to logger.exception for full tracebacks
- Include exception type in error messages for debugging

Testing:
- Added test_hitl.py with pytest unit tests
- Tests cover data models, mock model inference, and integration
- Includes fixtures for reusable test data

Variable Naming:
- Renamed loop variable 'f' to 'farm_name' for clarity
@shkurapa shkurapa reopened this Feb 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants