[lhotse] Added support for re-weighting datasets with temperature on the fly.

[XuesongYang]

Summary

This PR introduces temperature-based dataset reweighting to the Lhotse dataloader, enabling dynamic adjustment of sampling probabilities across datasets without manually recalculating weights. It removes the need to update dataset weights when adding or removing datasets. Simply specify the desired temperature, and the weights are automatically adjusted at runtime.

Key Changes

1. New temperature_reweighting() function

Applies temperature scaling to dataset weights using the formula:

ŵᵢ = wᵢ^τ / Σⱼ wⱼ^τ.

• τ = 1.0: Preserves original weight ratios (neutral)
• τ = 0.0: Equalizes all datasets regardless of original weights
• 0 < τ < 1.0: Over-samples smaller datasets relative to larger ones

2. Flexible reweight_temperature config option

Supports multiple input formats with automatic normalization:

• Scalar: Broadcasts to all nesting levels (with warning)
• List matching depth: Applied per-level as specified
• List too short: Extended by repeating last value (with warning)
• List too long: Trimmed to max depth (with warning)

Example - preserve top-level ratios but equalize within sub-groups:

reweight_temperature: [1.0, 0.0]  # Level 1: preserve, Level 2: equalize

3. Comprehensive test coverage

• Unit tests for temperature_reweighting() and count_input_cfg_levels()
• Integration tests for dataloader with various temperature configurations
• Validation tests for scalar/list normalization behavior

Documentation

See the new "Dataset Reweighting with Temperature" section in the audio configuration docs: docs/source/audio/configs.rst

[Copilot]

Pull request overview

This PR introduces temperature-based dataset reweighting to the Lhotse dataloader, enabling dynamic adjustment of sampling probabilities across datasets through a configurable temperature parameter. This eliminates the need for manual weight recalculation when combining datasets with different sizes.

Key changes:

• New temperature_reweighting() function that applies temperature scaling to weights using the formula (w_i ^ temp) / sum(w_j ^ temp)
• New reweight_temperature configuration option that supports hierarchical temperature application across nested dataset groups
• Comprehensive test suite with 19 tests covering various input types and edge cases

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File	Description
nemo/collections/common/data/lhotse/cutset.py	Adds the temperature_reweighting() function and integrates it into the dataset loading pipeline with temperature propagation through nested groups
examples/tts/conf/magpietts/magpietts_lhotse.yaml	Adds example configuration demonstrating hierarchical temperature usage with [1.0, 0.0]
tests/collections/common/test_lhotse_temperature_reweighting.py	Adds comprehensive unit and integration tests for the temperature reweighting functionality

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

nemo/collections/common/data/lhotse/dataloader.py:19

• Import of 'List' is not used.

from typing import Any, Optional, Sequence, Union

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[subhankar-ghosh]

This is my understanding, correct me if I am wrong: Level 1 (Language): preserves the weight, Level 2 (datasets within each language): equalizes.
Based on this understanding, don't we want to normalize the weights based on number of shar files/size of the datasets at level 2?

[XuesongYang]

you're right. But this is an example demonstrating how to use this new param. In practice, we should override this one based on your needs.