How to configure IPPO

[JulienHansen]

Hi,

I have a question regarding the YAML configuration for IPPO in SKRL. I'm working on a two-drone environment where one drone is tasked with reaching a target while the other must follow it. The first drone is behaving as expected. However, when I tested IPPO before implementing the follower logic, I expected the follower drone to essentially do nothing—but instead, it learns to fly.

I believe the issue is that the actor is shared between them and thus I need to configure the YAML file so that each agent has a different network architecture. Could someone please clarify how to set up the YAML configuration for IPPO in SKRL when I want my agents to use distinct model architectures? Here are my default IPPO config and the new one :

seed: 42


# Models are instantiated using skrl's model instantiator utility
# https://skrl.readthedocs.io/en/latest/api/utils/model_instantiators.html
models:
  separate: True # Separate models for policy and value
  policy:  # see gaussian_model parameters
    class: GaussianMixin
    clip_actions: False
    clip_log_std: True
    min_log_std: -20.0
    max_log_std: 2.0
    initial_log_std: 0.0
    network:
      - name: net
        input: STATES
        layers: [512, 256, 128]
        activations: elu
    output: ACTIONS
  value:  # see deterministic_model parameters
    class: DeterministicMixin
    clip_actions: False
    network:
      - name: net
        input: STATES
        layers: [512, 256, 128]
        activations: elu
    output: ONE

models:
  drone_attack:
    policy:
      class: GaussianMixin
      clip_actions: False
      clip_log_std: True
      min_log_std: -20.0
      max_log_std: 2.0
      initial_log_std: 0.0
      network:
        - name: net
          input: STATES
          layers: [512, 256, 128]
          activations: elu
      output: ACTIONS
    value:
      class: DeterministicMixin
      clip_actions: False
      network:
        - name: net
          input: STATES
          layers: [512, 256, 128]
          activations: elu
      output: ONE
  drone_defense:
    policy:
      class: GaussianMixin
      clip_actions: True
      clip_log_std: False
      min_log_std: -10.0
      max_log_std: 3.0
      initial_log_std: 0.5
      network:
        - name: net
          input: STATES
          layers: [256, 256]
          activations: relu
      output: ACTIONS
    value:
      class: DeterministicMixin
      clip_actions: False
      network:
        - name: net
          input: STATES
          layers: [256, 256]
          activations: relu
      output: ONE

of course the second one is not correct with skrl format it's just to express that both agent could have different model architecture

[RandomOakForest]

Thanks for posting this. @Toni-SM for vis.

[JulienHansen]

I found my answer in the skrl repository. Currently, in the .yaml file, it is not possible to directly specify different architectures (see: GitHub discussion)