Adds perceptive actor-critic class #114
Conversation
ClemensSchwarke
force-pushed
the
feature/perceptive-nav-rl
branch
2 times, most recently
from
22cd1ec to
8a0a959
Compare
rsl_rl/networks/cnn.py
Outdated
| if self.flatten: | ||
| x = x.flatten(start_dim=1) | ||
| elif self.avgpool is not None: | ||
| x = self.avgpool(x) | ||
| x = x.flatten(start_dim=1) | ||
| return x |
There was a problem hiding this comment.
I don't get this logic. Why do we always flatten when we do avgpool and why do we not do avgpool when we also set the flatten flag?
ClemensSchwarke
force-pushed
the
fix/recurrent_symmetry
branch
from
61038de to
d69435a
Compare
ClemensSchwarke
force-pushed
the
feature/perceptive-nav-rl
branch
from
42e406d to
df9e1ad
Compare
ClemensSchwarke
force-pushed
the
fix/recurrent_symmetry
branch
from
d69435a to
c871703
Compare
ClemensSchwarke
force-pushed
the
feature/perceptive-nav-rl
branch
from
df9e1ad to
0a1756d
Compare
|
@pascal-roth I modified the CNN quite a bit to have more flexibility. Could you quickly check if this version works for you and maybe also give it a quick check? I also modified your perceptive environment configs to work with the changes here, it would be awesome if you could run the env to see if it still trains the same :) |
|
Hello! I was following the progress of this PR and I had a question whether it makes sense to extend this ActorCriticPerceptive to have a shared CNN backbone for both critic and actor? Do you think it should be an option or should it be needed at all? I thought it can greatly simplify the complexity of neural network and speed up the training process. |
|
Hi @lvjonok, I would assume the speedup is negligible. But if you want to test this please share the results :) |
|
Hi @ClemensSchwarke in my tests, it seems using a shared CNN backbone between actor and critic can help save VRAM. I find this particularly helpful to use more environments per GPU |
Adds a new perceptive actor-critic class, that can define CNN layers for every 2D observation term.
Example usage shown in IsaacLab isaac-sim/IsaacLab#3467