Skip to content

Inconsistent Results with Rbeast #53

@anklesupport

Description

@anklesupport

In my experiments, I processed data using Rbeast’s Python interface with python 3.8 on a PC1 (Windows/Intel CPU). To ensure reproducibility, I explicitly set mcmc_seed as a positive integer and observed consistent results across repeated runs, which aligned with expectations.

However, when executing the identical code on another PC​2 (Windows/AMD CPU) with mcmc_seed configured, the results diverged among different runs with same inputs.

Notably, the result appeared distinct from the distribution observed when mcmc_seed=0 was used, not every result from different runs is different, but sometimes. Therefore, it doesn't seem like a problem of random number generation.

To eliminate environment-specific factors, I containerized the workflow in PC1 into a Docker image and ran it on PC2. The same inconsistency persisted. Further tests across additional OS/hardware revealed:

  • Inconsistent results were observed on PC2 (Windows/AMD CPU), PC3 (Windows/Intel CPU).
  • Consistent results were observed on PC1 (Windows/Intel CPU), PC4 (Linux/Intel CPU).

Environment & Code Details are as following:

​Docker: 25.0.3​
Python: 3.8.20
Numpy: 1.24.4
Rbeast: 0.1.23​

Input data: 500-length floating-point time series
Code:

o = rb.beast(  
    data_series,  
    season='none',  
    quiet=1,  
    mcmc_seed=12,  
    mcmc_samples=20000,  
    mcmc_chains=10  
)  

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions