Skip to content

Commit 3c25bb4

Browse files
committed
unify schemas, update pipes
1 parent bf498c0 commit 3c25bb4

40 files changed

+134
-487
lines changed
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
TOKEN "append" APPEND
2+
3+
# Refer to https://github.com/auth0/auth0-log-schemas and https://auth0.com/docs/deploy-monitor/logs/log-event-type-codes for the type codes
4+
5+
TAGS "auth0"
6+
7+
SCHEMA >
8+
`event_time` DateTime64(3) `json:$.tinybirdIngestTime` DEFAULT now(),
9+
`event_type` String `json:$.data.type` DEFAULT 'unknown',
10+
`event` JSON `json:$` DEFAULT '{}'
11+
12+
ENGINE "MergeTree"
13+
ENGINE_PARTITION_KEY "toYYYYMM(event_time)"
14+
ENGINE_SORTING_KEY "event_time"

tinybird/datasources/auth0_log_types.datasource

Lines changed: 0 additions & 12 deletions
This file was deleted.

tinybird/datasources/auth0_logs.datasource

Lines changed: 0 additions & 35 deletions
This file was deleted.

tinybird/datasources/clerk.datasource

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,10 @@ TOKEN "append" APPEND
55
TAGS "clerk"
66

77
SCHEMA >
8-
`time` DateTime64(3) `json:$.timestamp` DEFAULT now(),
9-
`data` JSON `json:$.data`,
10-
`event_attributes` JSON `json:$.event_attributes`,
11-
`object` String `json:$.object`,
12-
`type` String `json:$.type`
8+
`event_time` DateTime64(3) `json:$.tinybirdIngestTime` DEFAULT now(),
9+
`event_type` String `json:$.type` DEFAULT 'unknown',
10+
`event` JSON `json:$` DEFAULT '{}'
1311

1412
ENGINE "MergeTree"
15-
ENGINE_PARTITION_KEY "toYear(time)"
16-
ENGINE_SORTING_KEY "time"
13+
ENGINE_PARTITION_KEY "toYYYYMM(event_time)"
14+
ENGINE_SORTING_KEY "event_time"

tinybird/datasources/clerk_simple.datasource

Lines changed: 0 additions & 13 deletions
This file was deleted.

tinybird/datasources/fixtures/auth0_log_types.ndjson

Lines changed: 0 additions & 110 deletions
This file was deleted.

tinybird/datasources/fixtures/clerk_simple.ndjson

Lines changed: 0 additions & 1 deletion
This file was deleted.

0 commit comments

Comments
 (0)