-
Notifications
You must be signed in to change notification settings - Fork 8
Labels
bugSomething isn't workingSomething isn't workingenhancementNew feature or requestNew feature or request
Description
Problem:
-
When a seed is provided, the package resets the random number generator (RNG) states multiple times within a single campaign:
- The optimizer, model, designer, and DOE components each reset RNG states independently.
- During a normal workflow (without explicitly defining the optimizer/designer), random seed reset happens when
- the optimizer is initialized (torch and numpy)
- the designer is initialized (torch only)
- if DOE is used (numpy only)
- every iteration when each model is instantiated (torch and numpy)
-
When no seed is provided, there is no RNG control at all, resulting in no reproducibility.
Impact:
Repeated RNG resets cause correlated and repeated noise that biases results. In the model fitting stage, the model and scipy/Adam optimizer will always start from the same initial conditions. Additionally, global seed control is fragile, as unexpected random draws between steps break reproducibility unless perfectly replicated.
Proposed Fix:
- Centralize RNG state management to avoid multiple independent resets.
- Ensure the simulator noise RNG is initialized once per campaign and not reset every iteration.
- Provide consistent RNG control whether or not a seed is specified, enabling reproducibility.
- Refactor the code to use a single RNG state or RNG generator instance passed explicitly to components that require randomness.
- Add a package level toggle to switch back to current behavior for backward compatibility.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't workingenhancementNew feature or requestNew feature or request