I've been looking through the histogram code and have a question/concern about the variance estimation of the binned free energies by bootstrapping.
In generate_fes_histogram, we calculate an array of bin free energies f_i by looping over labels in bin_label, selecting configurations that fall into the given bin, and computing their combined log weight.
But bin_label only includes the bins occupied by our x_n sample, which in the case of bootstrapping might not cover all bins occupied by the original data. This means that our f_i arrays calculated for different bootstrapping runs could have different sizes or might not line up correctly (for example, f_i[0] for one bootstrap could refer to a different bin than f_i[0] for a different bootstrap, if that bin is occupied in one bootstrap but not the other). We would then get errors or incorrect results when computing the standard deviation here.
Does it make sense to replace bin_label/bin_label.values() with bin_order?
# unlike bin_label, bin_order always contains all bins occupied by the original dataset
f_i = np.zeros(len(bin_order), np.float64) # changed from bin_label
for i, label in enumerate(bin_order): # changed from bin_label.values()
...
# Compute dimensionless free energy of occupying state i.
f_i[bin_order[label]] = -logsumexp(log_w_nb[indices])
# store the free energies for this bin
histogram_data["f"] = f_i
This way, the f_i arrays for bootstrapping would always line up with each other and with the original data. For some bins and some bootstrapping samples, we could have an undefined/infinite free energy resulting from a bin occupancy of 0, which would lead to an undefined/infinite bootstrapping-estimated variance for that bin. This seems like reasonable behavior to me, but I'm not very familiar with bootstrapping conventions/best practices.
I've been looking through the histogram code and have a question/concern about the variance estimation of the binned free energies by bootstrapping.
In
generate_fes_histogram, we calculate an array of bin free energiesf_iby looping over labels inbin_label, selecting configurations that fall into the given bin, and computing their combined log weight.But
bin_labelonly includes the bins occupied by ourx_nsample, which in the case of bootstrapping might not cover all bins occupied by the original data. This means that ourf_iarrays calculated for different bootstrapping runs could have different sizes or might not line up correctly (for example,f_i[0]for one bootstrap could refer to a different bin thanf_i[0]for a different bootstrap, if that bin is occupied in one bootstrap but not the other). We would then get errors or incorrect results when computing the standard deviation here.Does it make sense to replace
bin_label/bin_label.values()withbin_order?This way, the
f_iarrays for bootstrapping would always line up with each other and with the original data. For some bins and some bootstrapping samples, we could have an undefined/infinite free energy resulting from a bin occupancy of 0, which would lead to an undefined/infinite bootstrapping-estimated variance for that bin. This seems like reasonable behavior to me, but I'm not very familiar with bootstrapping conventions/best practices.