Skip to content

Conversation

@vredchenko
Copy link
Collaborator

Summary

Draft implementation enabling SmartEM to log ML prediction decisions for Athena Decision Service API without database schema changes.

Changes

  • Extended AthenaClient with bulk decision submission (record_decisions method)
  • Added get_athena_session_id() helper (returns dummy UUID)
  • Modified decision helpers to log instead of POST
  • Enabled decision submission in prediction handlers:
    • handle_gridsquare_model_prediction
    • handle_foilhole_model_prediction
    • handle_multi_foilhole_model_prediction
  • Uses natural IDs (gridsquare_id, foilhole_id) as Athena area IDs

Approach

  • No database schema changes
  • No area registration (assumes external system handles this)
  • Dummy session ID: 00000000-0000-0000-0000-000000000000
  • Logs decision JSON for testing

Next Steps

  • Replace dummy session ID with actual EPU/database value
  • Implement Athena mock server
  • Enable actual HTTP calls
  • Verify with production Athena instance

@vredchenko vredchenko force-pushed the agent_athena_api_integration_draft branch from 36e2c16 to 0e4e9cb Compare November 4, 2025 17:59
@vredchenko vredchenko added the smartem agent Tasks related to the agent (data intake) component label Nov 4, 2025
…ntation

This commit enables SmartEM to prepare and log decisions that would be sent to the Athena Decision Service API, without requiring database schema changes or area registration infrastructure.

Changes:
- Add bulk decision submission support to AthenaClient (record_decisions method)
- Add get_athena_session_id helper function (returns dummy UUID for now)
- Modify decision submission functions to log instead of POST
- Enable decision submission in gridsquare prediction handler
- Enable decision submission in foilhole prediction handlers (single and bulk)
- Use natural IDs (gridsquare_id, foilhole_id) as Athena area IDs

Implementation approach:
- No database schema changes required
- No area registration (assumes EPU/microscope handles it)
- Uses dummy session ID (00000000-0000-0000-0000-000000000000)
- Logs decision JSON instead of making HTTP calls

This is a draft implementation for testing the decision flow. Future work:
- Replace dummy session ID with actual session from database/EPU
- Implement Athena mock server for testing
- Enable actual HTTP POST calls to Athena
- Verify area ID mapping with production Athena instance
@vredchenko vredchenko force-pushed the agent_athena_api_integration_draft branch from 0e4e9cb to f5ce77d Compare December 3, 2025 16:17
The Athena Decision Service integration was incorrectly placed in the
backend (consumer.py). This violates the separation of concerns:
- Backend handles recommendations (ML predictions, SSE routing)
- Agent handles instructions (user confirmation, microscope control)

Changes:
- Remove athena_api imports from consumer.py
- Remove athena_client initialization and helper functions
- Remove Athena calls from prediction handlers
- Delete WIP documentation file

Bug fixes retained from original PR:
- Fix CurrentQualityPrediction.value assignment (was overwriting object)
- Fix session.add_all() for list of predictions (was using add())

Agent-side Athena integration tracked in issue #198.
@vredchenko vredchenko changed the title [WIP] Athena Decision Service integration Athena Decision Service integration Dec 3, 2025
@vredchenko vredchenko merged commit 5967ec7 into main Dec 3, 2025
9 checks passed
@vredchenko vredchenko deleted the agent_athena_api_integration_draft branch December 3, 2025 18:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

smartem agent Tasks related to the agent (data intake) component

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants