Skip to content

LOGC-46: Fix DateTime64 millisecond truncation in LogFetcher#95

Merged
dvasilas merged 1 commit intomainfrom
bugfix/LOGC-46-2
Jan 23, 2026
Merged

LOGC-46: Fix DateTime64 millisecond truncation in LogFetcher#95
dvasilas merged 1 commit intomainfrom
bugfix/LOGC-46-2

Conversation

@dvasilas
Copy link
Collaborator

This substitutes #90 with a much simpler fix.

The issue is that the ClickHouse Go driver truncates milliseconds when binding time.Time to DateTime64(3) parameters.
I wrote a simple script that reproduces the issue
With the following table:

CREATE TABLE %s (
             id        String,
             startTime DateTime64(3)
) ENGINE = MergeTree() ORDER BY startTime

when we insert with

records := []struct {
        id        string
        startTime time.Time
    }{
        {"rec1", time.Date(2026, 1, 20, 16, 20, 36, 5*int(time.Millisecond), time.UTC)},   // 36.005s
        {"rec2", time.Date(2026, 1, 20, 16, 20, 36, 54*int(time.Millisecond), time.UTC)},  // 36.054s - THE OFFSET
        {"rec3", time.Date(2026, 1, 20, 16, 20, 36, 100*int(time.Millisecond), time.UTC)}, // 36.100s
    }

    for _, rec := range records {
        err := conn.Exec(ctx, fmt.Sprintf("INSERT INTO %s VALUES (?, ?)", tableName), rec.id, rec.startTime)
        if err != nil {
            t.Fatalf("failed to insert %s: %v", rec.id, err)
        }
    }

we get

docker exec workbench-clickhouse-shard-1 clickhouse-client --port 9002 -q "SELECT * FROM datetime64_test_1769182943901284695"
rec2    2026-01-20 16:20:36.000
rec3    2026-01-20 16:20:36.000
rec1    2026-01-20 16:20:36.000

but when we also insert

nowStr := fmt.Sprintf("%.3f", float64(time.Now().UnixMilli())/1000.0)
t.Logf("Inserting rec_str with timestamp string: %s", nowStr)
err = conn.Exec(ctx, fmt.Sprintf("INSERT INTO %s VALUES (?, ?)", tableName), "rec_str", nowStr)

we get

rec1    2026-01-20 16:20:36.000
rec2    2026-01-20 16:20:36.000
rec3    2026-01-20 16:20:36.000
rec_now 2026-01-23 15:52:48.000
rec_str 2026-01-23 15:52:48.815

The fix is to apply this conversion to log-courier

The ClickHouse Go driver truncates milliseconds when binding time.Time
to DateTime64(3) parameters.
Convert startTime to Unix epoch string format to preserve millisecond
precision in queries.
@dvasilas dvasilas merged commit f4cb6cc into main Jan 23, 2026
5 checks passed
@dvasilas dvasilas deleted the bugfix/LOGC-46-2 branch January 23, 2026 17:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants