Refactor Configuration Structure: Move Service Configs to Top-Level and Simplify Policy Initialization

[DNXie]

As discussed in PR #153

• Moved service-related configuration from under policy to a new top-level services section in the config file.
• Updated code accordingly.
• Removed the exclude_service function and all related usages.
• Fixed a log printing issue from Refactor service spawning: add ForgeActor.options().as_service() API #153

Before

trainer:
  model_name: ${model}
  learning_rate: 1e-5
  service:
    procs_per_replica: 1
    num_replicas: 1
    with_gpus: true

Trainer.options(**cfg.trainer.service).as_service(**exclude_service(cfg.trainer))

After

trainer:
  model_name: ${model}
  learning_rate: 1e-5


services:
  trainer:
    procs_per_replica: 1
    num_replicas: 1
    with_gpus: true

Trainer.options(**cfg.services.trainer).as_service(**cfg.trainer)

Test

python -m apps.vllm.main --config apps/vllm/llama3_8b.yaml
python -m apps.vllm.main --config apps/vllm/qwen2_5_32b.yaml
python -m apps.vllm.main --config apps/vllm/deepseek_r1
python -m apps.grpo.main --config apps/grpo/qwen3_1_7b.yaml
python -m apps.grpo.main --config apps/grpo/qwen3_multinode.yaml

[DNXie]

@allenwang28 When I was testing

python -m apps.grpo.main --config apps/grpo/qwen3_multinode.yaml

Because the config specifies hosts_per_replica: 1, I got the error saying

KeyError: "No named resource found for `gpu.small`. Registered named resources: ['NULL', 'MISSING']"

I had to change this line from

image="test", meshes=[f"{name}:{num_hosts}:gpu.small"]

to

image="test", meshes=[f"{name}:{num_hosts}:NULL"]

to make it work.

Does it indicate any errors with my local resource allocation?

[allenwang28]

Re your question, I would expect that the multi-host doesn't run given that you're not running on the SLURM cluster