-
-
Notifications
You must be signed in to change notification settings - Fork 9
Description
The Date and Timestamp functions are currently normalized to milliseconds.
This makes it easy to reason about parsing and creating these objects predictably, and it pushes unit conversion to the user. Once you understand it, the library becomes simple and consistent — predictable conversions, clear errors for misaligned inputs, and fewer surprises.
However, in version 2.2.4, when working with the Time type, this pattern breaks: time* values require absolute unit-based inputs.
Once you know it, it’s manageable, but it creates a worse developer experience to have two different systems. It also increases the chance of subtle bugs. For example, fractional Number inputs are currently accepted for time due to auto-inferred bitWidth (which [#51] should resolve).
const nanosWrong = columnFromArray([(3600 * 1000) + 0.000001], time(TimeUnit.NANOSECOND));
const nanosRight = columnFromArray([3600 * 1000 * 1000 * 1000], time(TimeUnit.NANOSECOND, 64));
const td = tableFromColumns([['nanosWrong', nanosWrong], ['nanosRight', nanosRight]]);
console.log(td.toArray());
// => [{"nanosWrong":3600000,"nanosRight":3600000000000}]Proposal
Normalize all time-related input in the library to milliseconds, matching how Date and Timestamp already behave.
This would make reasoning about temporal data much simpler and the API more consistent.
It’s a tradeoff between API consistency and explicit user control over rounding/truncation.
The Number type has 53 bits of precision — 1 bit for sign, leaving 52 usable.
Representing a full day in nanoseconds only needs 47 bits, so this approach works without precision loss.
Because this breaks the current API, it would be suited for a v3 proposal.
I can take this on if we all agree it’s the right direction. For now, I’ll add helper functions to handle the conversion and ensure reliable input behavior.
Example Implementation could look something like this
We could do something similar to toTimestamp:
export function toTime(unit) {
return unit === TimeUnit.SECOND ? value => (value / 1e3) | 0
: unit === TimeUnit.MILLISECOND ? value => value | 0
: unit === TimeUnit.MICROSECOND ? value => toBigInt(value * 1e3)
: value => toBigInt(value * 1e6);
}And then in src/build/builder.js:
case Type.Time:
return new TransformBuilder(type, ctx, toTime(type.unit));