Maybe a bug in DDIM and results

[LenarBe]

Dear suchen4565:
Thank you for your impressive work! I am trying to reproduce the result in the paper, and the results in Health(US) in the 96–12 setting got wrong. I found there may be a few issues confusing.

1. In DDIM sampling, commonly the referenced t in the model should be the transformed tau in sampling, but the code performs differently.
2. In the results, there are MSE and NMSE metrics, and I do not know which metrics are reported.
3. Finally, I utilize the default configs in experiment and get a result of MSE 1.860 and NMSE 1.274, which is quite different from the reports. Is the language model utilized bert here? And is the ddim used by default?
   Thank you if you could respond to me. I am looking forward to your reply!

[suchen4565]

Thank you for your interest in our work and for sharing your feedback. Below are clarifications addressing your questions:

1. Our code indeed uses the transformed tau (derived from t) during sampling, as implemented in main_model.py.
2. In our paper, the reported MSE and MAE results correspond to the normalized MSE/MAE (NMSE/NMAE) computed after dataset normalization. This was referred to as NMSE/NMAE in earlier versions of our code.
3. The MSE/MAE results in the paper represent averaged values across runs. Standard deviations for these metrics are provided in the appendix of the paper.

We recently updated the repository with refined configurations and code. We strongly recommend using the latest version.

[LenarBe]

Thank you for your reply! For the first issue, I am not sure if I had made a mistake, but you can check in the impute function, where the ddim is used. The prediction here I think should be the refinement of previous result, with the diffusion step passed to the model as tau, but in the prediction is actually inferenced as diffusion step t.