Skip to content

Design: Treat all task return values as tuples for consistency #241

@coderabbitai

Description

@coderabbitai

Background

During the review of PR #240, a design-level discussion emerged about how task return values should be handled in Spider's Python (and potentially C++) API.

Current Behavior

Currently, Spider differentiates between:

  • Single return values: returned as-is
  • Multiple return values: returned as tuples

This mirrors Python's native function call behavior.

Proposed Change

Treat all return values as tuples, including single return values (as single-element tuples). This would mean:

  • Single return value: (result,) = task.get_result()
  • Multiple return values: (result1, result2) = task.get_result()

Motivation

@LinZhihao-723 observed that making tasks behave like normal Python function calls may not be as ergonomic as initially thought, especially considering the full lifecycle of task submission and result retrieval. The current approach of mirroring Python's function return scheme adds complexity without necessarily improving usability.

Reference: CLP compression integration example at https://github.com/y-scope/clp/blob/main/components/job-orchestration/job_orchestration/scheduler/compress/task_manager/spider_task_manager.py

Scope

This change would affect:

  • Python API: task result retrieval
  • Potentially C++ API (though C++ uses templates which provide compile-time validation)

Discussion Reference

Original discussion: #240 (comment)

Participants: @LinZhihao-723, @sitaowang1998

cc: @LinZhihao-723

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions