-
Notifications
You must be signed in to change notification settings - Fork 9
Description
Background
During the review of PR #240, a design-level discussion emerged about how task return values should be handled in Spider's Python (and potentially C++) API.
Current Behavior
Currently, Spider differentiates between:
- Single return values: returned as-is
- Multiple return values: returned as tuples
This mirrors Python's native function call behavior.
Proposed Change
Treat all return values as tuples, including single return values (as single-element tuples). This would mean:
- Single return value:
(result,) = task.get_result() - Multiple return values:
(result1, result2) = task.get_result()
Motivation
@LinZhihao-723 observed that making tasks behave like normal Python function calls may not be as ergonomic as initially thought, especially considering the full lifecycle of task submission and result retrieval. The current approach of mirroring Python's function return scheme adds complexity without necessarily improving usability.
Reference: CLP compression integration example at https://github.com/y-scope/clp/blob/main/components/job-orchestration/job_orchestration/scheduler/compress/task_manager/spider_task_manager.py
Scope
This change would affect:
- Python API: task result retrieval
- Potentially C++ API (though C++ uses templates which provide compile-time validation)
Discussion Reference
Original discussion: #240 (comment)
Participants: @LinZhihao-723, @sitaowang1998
cc: @LinZhihao-723