resample: ValueError: x and y arrays must be equal in length along interpolation axis.

[adelavega]

When trying to resample a dense variable (physio) with a sampling rate of 0.49999984750004656
the length of the x given to interp1d is one longer than values. My guess is that this has to do with a rounding error with the funky sampling_rate.

[adelavega]

Actually, looks like the issue is len(self.index) != len(self.values) for that variable. Not sure why though.

[adelavega]

This traces back to _build_entity_index.
When calculating the index, it uses the following formula:
reps = int(math.ceil(run.duration * sampling_rate))

In this dataset, with duration=876.00031328201294 and sampling_rate=500, that equals:
438000.15664100647 which math.ceil rounds to 438001.

The question is if that strong ceiling rounding is fair here. I could imagine why only 438000 samples are provided. Maybe regular np.round is more appropriate, or somehow this function can be sensitive to the actual number of samples?

[tyarkoni]

I think changing it to round is probably fine. But either way, what we should probably do is add a check (called after every update or transformation) that makes sure that all of the internal arrays have the same length (i.e., duration, onset, amplitude, and values) for sparse variables, and that the index has the same length as the values for dense ones.

[adelavega]

Agreed. But if its off by one? Should this check adjust it? I can just see either number of samples being valid (although round seems like a saner default to me if we had to choose).

[tyarkoni]

Yes, the idea would be to trim or pad one or the other (or at the very least, raise a more comprehensible exception). Probably the values should take priority and the index adjusted to respect its dimensions.

[effigies]

Possibly related to #355, #358?

[adswa]

I believe I ran into this (and referenced it incoherently in the lower end of #355)

[adelavega]

@effigies try changing the rounding from math.ceil to round in your PR (in _build_entity_index. but I think it occurs in at least one other place) to see if that fixes your issue. For issues such as mine (undetermined actual number of volumes, in between whole numbers), we'd need to implement the function that evens out the # of vols.

[effigies]

f936aed

[adelavega]

This was fixed by #365

[adelavega]

This is coming up for me again, in the context of densifying a sparse variable after HRF (in #411), for a dataset with a TR of 2.00000061.

Anyone else have seen this issue crop up again? I thought #365 added a checkt o align the values even if there was a rounding mismatch