Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
4f2482e
Fix articulation data micro-benchmarks
AntoineRichard Jul 20, 2026
e3eb0e2
Add OVPhysX asset micro-benchmarks
AntoineRichard Jul 20, 2026
f3a0987
Add OVPhysX sensor micro-benchmarks
AntoineRichard Jul 16, 2026
8ffe3e0
Harden sensor micro-benchmark timing
AntoineRichard Jul 20, 2026
6ecdc8c
Add Newton sensor micro-benchmarks
AntoineRichard Jul 16, 2026
088147d
Standardize Newton sensor benchmarks
AntoineRichard Jul 20, 2026
e26db4c
Document micro-benchmark workflows
AntoineRichard Jul 17, 2026
55ff4e2
Clarify benchmark publication provenance
AntoineRichard Jul 20, 2026
d5c93a9
Reorganize benchmark documentation
AntoineRichard Jul 17, 2026
89a40e2
Use standard benchmark output examples
AntoineRichard Jul 17, 2026
db2bd43
Rewrite benchmark user guide
AntoineRichard Jul 17, 2026
0af66f8
Explain startup iteration cost
AntoineRichard Jul 17, 2026
14c5264
Fix benchmark schema provenance
AntoineRichard Jul 17, 2026
f39c303
Add benchmark workflow examples
AntoineRichard Jul 17, 2026
63c8f81
Rewrite advanced benchmark framework guide
AntoineRichard Jul 20, 2026
caeda2c
Unify benchmark documentation navigation
AntoineRichard Jul 28, 2026
0f71c33
Add training success-rate curves
AntoineRichard Jul 20, 2026
63c5c82
Record training success-rate series
AntoineRichard Jul 20, 2026
cd256b4
Simplify benchmark success smoke checks
AntoineRichard Jul 20, 2026
e5240d7
Use logged success series in bundles
AntoineRichard Jul 20, 2026
7242587
Fix success tracker documentation
AntoineRichard Jul 28, 2026
f0b30fe
Restore RL entrypoint backend state
AntoineRichard Jul 20, 2026
bb7cee1
Refine RL entrypoint state restoration
AntoineRichard Jul 21, 2026
3627310
Standardize benchmark warm-up steps
AntoineRichard Jul 28, 2026
15b1d9c
Bump success series schema version
AntoineRichard Jul 28, 2026
ca1a3fd
Merge benchmark success series
AntoineRichard Jul 28, 2026
c304c58
Merge RL entrypoint state restoration
AntoineRichard Jul 28, 2026
112c255
Merge benchmark warm-up step standardization
AntoineRichard Jul 28, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions docs/source/api/lab/isaaclab.benchmark.rst
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,41 @@ isaaclab.benchmark
MethodBenchmarkDefinition
MethodBenchmarkRunner
MethodBenchmarkRunnerConfig

.. rubric:: Measurements and metadata

.. autosummary::

BooleanMeasurement
DictMeasurement
DictMetadata
FloatMetadata
IntMetadata
ListMeasurement
Measurement
MetadataBase
SingleMeasurement
StatisticalMeasurement
StringMetadata
TestPhase

.. rubric:: Result schema

.. autosummary::

EnvironmentStepTiming
Learning
LearningCurve
MeanStd
PlayBundle
Resources
RunConfig
RunIdentity
Runtime
RuntimeBundle
StartupBundle
StartupPhase
StartupTime
TrainingBundle


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -125,27 +125,10 @@ CPU: Intel(R) Xeon(R) Platinum 8362 CPU @ 2.80GHz
+-------------------------------------+-------------------+--------------+-------------------+--------------------+

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can probably remove this whole file at this point



Benchmark Scripts
-----------------

For ease of reproducibility, we provide benchmarking scripts available at ``scripts/benchmarks``.
The unified entry points cover RL training with any supported library as well as environment
stepping without any reinforcement learning library.

Run the benchmark entry points through the Isaac Lab CLI:

.. code-block:: bash

# benchmark with RSL RL
uv run isaaclab benchmark training --rl_library rsl_rl --task=Isaac-Cartpole

# benchmark with RL Games
uv run isaaclab benchmark training --rl_library rl_games --task=Isaac-Cartpole

# benchmark without RL libraries (environment stepping only)
uv run isaaclab benchmark runtime --task=Isaac-Cartpole
Reproduce These Workloads
-------------------------

Each benchmark emits a schema-v1 JSON bundle at the end of the run, which includes data on the
startup times, runtime statistics such as the time taken for each simulation or rendering step,
as well as overall environment FPS for stepping the environment, performing inference during
rollout, and training.
Use the supported runtime, play, training, and startup workflows described in
:ref:`testing_benchmarks`. Record the task, backend, CPU, GPU, software revision,
environment count, seed, warm-up, and measured window with every published
result.
Loading
Loading