Skip to content

Improvement: Standardize and normalize all time-related input to milliseconds #52

@roelentless

Description

@roelentless

The Date and Timestamp functions are currently normalized to milliseconds.
This makes it easy to reason about parsing and creating these objects predictably, and it pushes unit conversion to the user. Once you understand it, the library becomes simple and consistent — predictable conversions, clear errors for misaligned inputs, and fewer surprises.

However, in version 2.2.4, when working with the Time type, this pattern breaks: time* values require absolute unit-based inputs.

Once you know it, it’s manageable, but it creates a worse developer experience to have two different systems. It also increases the chance of subtle bugs. For example, fractional Number inputs are currently accepted for time due to auto-inferred bitWidth (which [#51] should resolve).

const nanosWrong = columnFromArray([(3600 * 1000) + 0.000001], time(TimeUnit.NANOSECOND));
const nanosRight = columnFromArray([3600 * 1000 * 1000 * 1000], time(TimeUnit.NANOSECOND, 64));
const td = tableFromColumns([['nanosWrong', nanosWrong], ['nanosRight', nanosRight]]);
console.log(td.toArray());
// => [{"nanosWrong":3600000,"nanosRight":3600000000000}]

Proposal

Normalize all time-related input in the library to milliseconds, matching how Date and Timestamp already behave.
This would make reasoning about temporal data much simpler and the API more consistent.

It’s a tradeoff between API consistency and explicit user control over rounding/truncation.

The Number type has 53 bits of precision — 1 bit for sign, leaving 52 usable.
Representing a full day in nanoseconds only needs 47 bits, so this approach works without precision loss.

Because this breaks the current API, it would be suited for a v3 proposal.
I can take this on if we all agree it’s the right direction. For now, I’ll add helper functions to handle the conversion and ensure reliable input behavior.

Example Implementation could look something like this

We could do something similar to toTimestamp:

export function toTime(unit) {
  return unit === TimeUnit.SECOND ? value => (value / 1e3) | 0
    : unit === TimeUnit.MILLISECOND ? value => value | 0
    : unit === TimeUnit.MICROSECOND ? value => toBigInt(value * 1e3)
    : value => toBigInt(value * 1e6);
}

And then in src/build/builder.js:

case Type.Time:
  return new TransformBuilder(type, ctx, toTime(type.unit));

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions