Open
Conversation
This commit implements the feature requested in issue #295 to capture and retain stdout/stderr logs from ProtocolDAG executions and expose them via the AlchemiscaleClient. Changes: - Added ProtocolDAGResultLog model to store log references in the state store - Modified compute service to capture stdout/stderr during DAG execution - Updated compute client to send logs along with results - Updated compute API to receive and store logs in object store - Added state store methods to link logs to ProtocolDAGResults - Added object store methods to persist and retrieve logs - Implemented get_task_stdout/get_task_stderr client methods - Added API endpoint to retrieve task logs The implementation captures logs during execution, stores them as artifacts in S3, and provides easy access through new client methods for debugging failed or completed tasks.
- interface/api.py: Rewrite get_task_logs endpoint; the original called n4js.get_scoped_key() with wrong argument types and constructed ScopedKey with invalid positional args. Now retrieves S3 locations directly from statestore and passes them to objectstore. Use http_status constant instead of bare 400. - compute/api.py: Use `is not None` checks instead of truthiness for stdout/stderr to preserve empty string logs. - statestore.py: Replace f-string interpolation in Cypher query with parameterized $stream to prevent injection. Return S3 locations directly instead of ScopedKeys to avoid unnecessary round-trips. - objectstore.py: Simplify pull_protocoldagresult_log to accept only location (always available from statestore). Move ProtocolDAGResultLog import to module level. - compute/service.py: Use _TeeStream for stderr capture so that logging output from the StreamHandler is both captured and still emitted to the original stderr, preventing log suppression during DAG execution. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #295.
Changes:
ProtocolDAGResultLogmodel to store log references in the state storestdout/stderrduring DAG executionProtocolDAGResultsget_task_stdout/get_task_stderrclient methodsThe implementation captures logs during execution, stores them as artifacts in S3, and provides easy access through new client methods for debugging failed or completed tasks.