Commit 0dca89a
[Feature] AsyncBatchedCollector: coordinator loop and direct submission mode (#3499)
Rewrite the AsyncBatchedCollector to use a coordinator thread that
pipelines env stepping and batched inference without a global sync
barrier. Add a `direct=True` mode where each env thread submits
directly to the InferenceServer, eliminating the coordinator thread
and its serialization overhead.
Benchmark results (8 mock pixel envs, Nature-CNN, CPU):
AsyncBatchedCollector direct: 3183 fps (+72% vs coordinator)
AsyncBatchedCollector threading: 1850 fps (coordinator mode)
AsyncBatchedCollector mp: 1042 fps (coordinator mode)
Co-authored-by: Cursor <cursoragent@cursor.com>
ghstack-source-id: 861a24d
Pull-Request: #3499
Co-authored-by: Cursor <cursoragent@cursor.com>1 parent fe0bcfe commit 0dca89a
File tree
6 files changed
+750
-242
lines changed- benchmarks
- examples/collectors
- test
- torchrl
- collectors
- envs
6 files changed
+750
-242
lines changed
0 commit comments