[ez][CH] Add default.workflow_job schema (#6848)

This is a noop, like all other schemas, this may not reflect the actual
schema and is not actually enfored. It is here mainly to help with
documentation and to attempt to track changes

Add the schema so we can know what's there

It's mostly the webhook that github sends for a workflow_job plus a few
extra things:
Found in dynamo: 
* torchci_classification - holds the log classification from log
classifier
* backfill - a column that tells if this is eligible for backfill from
the backfill job

Aliases
* _inserted_at - pretty much what it says
* repository_full_name - ex pytorch/pytorch, comes from the html_url, so
we don't need to join with the workflow_run table to get this info
* workflow_event - similar to above but uses dicts to efficiently get
the information from workflow_run
* workflow_created_at - same as above

Keep going related things:
(kg stands for keep going)
* torchci_classification_temp - holds the log classifier result from the
temp log while the job does continue through error/keep going
* torchci_classification_kg - alias that combines
torchci_classification_temp and torchci_classification for easy querying
* conclusion_kg - similar to the above but for conclusion, basically
returns "failure, or will fail"
* log_url - alias for log url so we don't need to concat in queries all
the time, and also takes into account the temp log from keep going

Author: clee2000

clickhouse_db_schema/default.workflow_job/schema.sql

@@ -0,0 +1,68 @@
+CREATE TABLE default.workflow_job
+(
+    `check_run_url` String,
+    `completed_at` DateTime64(9),
+    `conclusion` String,
+    `created_at` DateTime64(9),
+    `dynamoKey` String,
+    `head_branch` String,
+    `head_sha` String COMMENT 'Contains commit SHA that matches workflow_run.head_commit.id',
+    `html_url` String,
+    `id` Int64,
+    `labels` Array(String),
+    `name` String,
+    `node_id` String,
+    `run_attempt` Int64,
+    `run_id` Int64,
+    `run_url` String,
+    `runner_group_id` Int64,
+    `runner_group_name` String,
+    `runner_id` Int64,
+    `runner_name` String,
+    `started_at` DateTime64(9),
+    `status` String,
+    `steps` Array(Tuple(
+        completed_at DateTime64(9),
+        conclusion String,
+        name String,
+        number Int64,
+        started_at DateTime64(9),
+        status String)),
+    `torchci_classification` Tuple(
+        captures Array(String),
+        context Array(String),
+        job_id Int64,
+        line String,
+        line_num Int64,
+        rule String),
+    `url` String,
+    `workflow_name` String,
+    `backfill` Bool DEFAULT true,
+    `_inserted_at` DateTime MATERIALIZED now(),
+    `repository_full_name` String ALIAS extractAll(url, 'https://api.github.com/repos/([^/]+/[^/]+)/actions/jobs/')[1] COMMENT 'Repository name in format "owner/repo" extracted from URL. Matches workflow_run.repository.full_name',
+    `workflow_event` String ALIAS dictGet('default.workflow_events_dict', 'event', dictGet('default.workflow_run_dict', 'event_hash', run_id)) COMMENT 'Corresponds to the event column in workflow_run table',
+    `workflow_created_at` DateTime ALIAS dictGet('default.workflow_run_dict', 'created_at', run_id) COMMENT 'Corresponds to the created_at column in workflow_run table',
+    `torchci_classification_temp` Tuple(
+        captures Array(String),
+        context Array(String),
+        job_id Int64,
+        line String,
+        line_num Int64,
+        rule String) COMMENT 'Holds the log classifier result from the temp log while the job does continue through error/keep going',
+    `torchci_classification_kg` Tuple(
+        captures Array(String),
+        context Array(String),
+        job_id Int64,
+        line String,
+        line_num Int64,
+        rule String) ALIAS if(tupleElement(torchci_classification, 'line') = '', torchci_classification_temp, torchci_classification) COMMENT 'Combined torchci_classification and torchci_classification_temp for easier querying for keep going.  kg stands for keep going',
+    `conclusion_kg` String ALIAS if((conclusion = '') AND (tupleElement(torchci_classification_temp, 'line') != ''), 'failure', conclusion) COMMENT 'Altered conclusion for keep going',
+    `log_url` String ALIAS multiIf(repository_full_name != 'pytorch/pytorch', concat('https://ossci-raw-job-status.s3.amazonaws.com/log/', repository_full_name, '/', CAST(id, 'String')), (tupleElement(torchci_classification_temp, 'line') != '') AND (conclusion = ''), concat('https://gha-artifacts.s3.us-east-1.amazonaws.com/temp_logs/', CAST(id, 'String')), concat('https://ossci-raw-job-status.s3.amazonaws.com/log/', CAST(id, 'String'))) COMMENT 'Log url for the job. Takes into account temp logs if possible',
+    INDEX status_index status TYPE bloom_filter GRANULARITY 1,
+    INDEX created_at_index created_at TYPE minmax GRANULARITY 1,
+    INDEX started_at_index started_at TYPE minmax GRANULARITY 1,
+    INDEX completed_at_index completed_at TYPE minmax GRANULARITY 1
+)
+ENGINE = SharedReplacingMergeTree('/clickhouse/tables/{uuid}/{shard}', '{replica}')
+ORDER BY (id, run_id, dynamoKey)
+SETTINGS index_granularity = 8192