File tree Expand file tree Collapse file tree 2 files changed +5
-6
lines changed Expand file tree Collapse file tree 2 files changed +5
-6
lines changed Original file line number Diff line number Diff line change @@ -54,8 +54,8 @@ class LLMEngine:
54
54
scheduler_config: The configuration related to the request scheduler.
55
55
distributed_init_method: The initialization method for distributed
56
56
execution. See `torch.distributed.init_process_group` for details.
57
- stage_devices: The list of devices for each stage. Each stage is a list
58
- of (rank, node_resource, device) tuples .
57
+ placement_group: Ray placement group for distributed execution.
58
+ Required for distributed execution .
59
59
log_stats: Whether to log statistics.
60
60
"""
61
61
Original file line number Diff line number Diff line change @@ -63,11 +63,10 @@ def initialize_cluster(
63
63
the default Ray cluster address.
64
64
65
65
Returns:
66
- A tuple of (`distributed_init_method`, `all_stage_devices `). The
66
+ A tuple of (`distributed_init_method`, `placement_group `). The
67
67
`distributed_init_method` is the address for initializing the
68
- distributed backend. `all_stage_devices` includes device IDs for
69
- each worker in each pipeline stage. Each device ID is a tuple of
70
- (rank, node resource, device id).
68
+ distributed backend. `placement_group` includes the specification
69
+ of the resources for each distributed worker.
71
70
"""
72
71
if parallel_config .worker_use_ray or engine_use_ray :
73
72
if ray is None :
You can’t perform that action at this time.
0 commit comments