[Question] Adding a wind field to crazyfly RL training code

[forwardyyy]

Question

Hello, I am now looking to add a wind field to the quadcopter reinforcement learning training process and implement it in python code. Here is the python code I see.

               # Create the force field prim
                xformPrim = UsdGeom.Xform.Define(stage, primPathName)
                xformPrim.AddTranslateOp().Set(Gf.Vec3f(0.0, 500.0, 0.0))
                prim = xformPrim.GetPrim()

                # Add the wind force fields to the prim
                windApi = ForceFieldSchema.PhysxForceFieldWindAPI.Apply(prim, "wind")
                windApi.CreateDragAttr(1000.0)
                windApi.CreateAverageSpeedAttr(500.0)
                windApi.CreateSpeedVariationAttr(2.0)
                windApi.CreateSpeedVariationFrequencyAttr(3.0)
                windApi.CreateAverageDirectionAttr(Gf.Vec3f(0.0, 0.0, 1.0))
                windApi.CreateDirectionVariationAttr(Gf.Vec3f(0.707, 0.707, 0.0))
                windApi.CreateDirectionVariationFrequencyAttr(Gf.Vec3f(2.0, 2.0, 2.0))
                forceFieldApi = ForceFieldSchema.PhysxForceFieldAPI(windApi, "wind")
                forceFieldApi.CreateEnabledAttr(True)
                forceFieldApi.CreatePositionAttr(Gf.Vec3f(0.0, 0.0, 0.0))
                forceFieldApi.CreateRangeAttr(Gf.Vec2f(-1.0, -1.0))

                forceFieldApi.CreateSurfaceSampleDensityAttr(surfaceSampleDensity)

But I don’t know where I should put them in quadcopter_env.py.

class QuadcopterEnvCfg(DirectRLEnvCfg):
    # env
    episode_length_s = 10.0
    decimation = 2
    action_space = 4
    observation_space = 12
    state_space = 0
    debug_vis = True

    ui_window_class_type = QuadcopterEnvWindow

    # simulation
    sim: SimulationCfg = SimulationCfg(
        dt=1 / 100,
        render_interval=decimation,
        disable_contact_processing=True,
        physics_material=sim_utils.RigidBodyMaterialCfg(
            friction_combine_mode="multiply",
            restitution_combine_mode="multiply",
            static_friction=1.0,
            dynamic_friction=1.0,
            restitution=0.0,
        ),
    )
    terrain = TerrainImporterCfg(
        prim_path="/World/ground",
        terrain_type="plane",
        collision_group=-1,
        physics_material=sim_utils.RigidBodyMaterialCfg(
            friction_combine_mode="multiply",
            restitution_combine_mode="multiply",
            static_friction=1.0,
            dynamic_friction=1.0,
            restitution=0.0,
        ),
        debug_vis=False,
    )

    # scene
    scene: InteractiveSceneCfg = InteractiveSceneCfg(num_envs=4096, env_spacing=2.5, replicate_physics=True)

    # robot
    robot: ArticulationCfg = CRAZYFLIE_CFG.replace(prim_path="/World/envs/env_.*/Robot")
    thrust_to_weight = 1.9
    moment_scale = 0.01

    # reward scales
    lin_vel_reward_scale = -0.05
    ang_vel_reward_scale = -0.01
    distance_to_goal_reward_scale = 15.0


class QuadcopterEnv(DirectRLEnv):
    cfg: QuadcopterEnvCfg

    def __init__(self, cfg: QuadcopterEnvCfg, render_mode: str | None = None, **kwargs):
        super().__init__(cfg, render_mode, **kwargs)

        # Total thrust and moment applied to the base of the quadcopter
        self._actions = torch.zeros(self.num_envs, gym.spaces.flatdim(self.single_action_space), device=self.device)
        self._thrust = torch.zeros(self.num_envs, 1, 3, device=self.device)
        self._moment = torch.zeros(self.num_envs, 1, 3, device=self.device)
        # Goal position
        self._desired_pos_w = torch.zeros(self.num_envs, 3, device=self.device)

        # Logging
        self._episode_sums = {
            key: torch.zeros(self.num_envs, dtype=torch.float, device=self.device)
            for key in [
                "lin_vel",
                "ang_vel",
                "distance_to_goal",
            ]
        }
        # Get specific body indices
        self._body_id = self._robot.find_bodies("body")[0]
        self._robot_mass = self._robot.root_physx_view.get_masses()[0].sum()
        self._gravity_magnitude = torch.tensor(self.sim.cfg.gravity, device=self.device).norm()
        self._robot_weight = (self._robot_mass * self._gravity_magnitude).item()

        # add handle for debug visualization (this is set to a valid handle inside set_debug_vis)
        self.set_debug_vis(self.cfg.debug_vis)

    def _setup_scene(self):
        self._robot = Articulation(self.cfg.robot)
        self.scene.articulations["robot"] = self._robot

        self.cfg.terrain.num_envs = self.scene.cfg.num_envs
        self.cfg.terrain.env_spacing = self.scene.cfg.env_spacing
        self._terrain = self.cfg.terrain.class_type(self.cfg.terrain)
        # clone, filter, and replicate
        self.scene.clone_environments(copy_from_source=False)
        self.scene.filter_collisions(global_prim_paths=[self.cfg.terrain.prim_path])
        # add lights
        light_cfg = sim_utils.DomeLightCfg(intensity=2000.0, color=(0.75, 0.75, 0.75))
        light_cfg.func("/World/Light", light_cfg)

Dear,
yoko

[Mayankm96]

The wind API doesn't support the parallelized Tensor API that we use in IsaacLab. So it would be hard to integrate the two for RL training. As an alternate, you could create your own model for wind simulation model and apply external forces on the bodies.

[forwardyyy]

Thank you very much for responding to me! Is there a document I can refer to about applying external forces on the bodies? I've searched everywhere for information, but I've come up empty-handed.┭┮﹏┭┮

[forwardyyy]

The wind API doesn't support the parallelized Tensor API that we use in IsaacLab. So it would be hard to integrate the two for RL training. As an alternate, you could create your own model for wind simulation model and apply external forces on the bodies.
When you say create my own model do you mean a drone model or a wind model? Is there a document I can refer to about applying external forces on the bodies? Thank you very much for responding to me!

[RandomOakForest]

Thanks for posting this and following up. Will move this post to our Discussions for follow up.