-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Description
Problem
Cube version: v1.3.9 (tried also 1.3.10, 1.3.11)
Getting TLSSocket.socketCloseListener ECONNRESET error on Cube API with ClickHouse as datasource during lambda execution here:
await pipeline(tableData.rowStream, writer, (err) => {
if (err) {
errors.push(err);
}
});
Error message (had to build own app with bunch of console.log messages to find out):
Error: aborted
at TLSSocket.socketCloseListener (node:_http_client:478:19)
at TLSSocket.emit (node:events:530:35)
at node:net:351:12
at TCP.done (node:_tls_wrap:650:7) {
code: 'ECONNRESET'
Acc to ClickHouse query_log query is Finished without any exceptions.
Problem observed only with query handler "stream" type (which seems set as stream by default for preaggregation lambdas - link).
If I remove lambda from schema or manually comment out queryHandler === 'stream' condition from QueryQueue.js, it works.
Example of schema:
cubes:
- name: cube1
sql: >
some query here
- name: cube2
sql: >
another query here
- name: cube_total
sql: >
{ cube1.sql() }
UNION ALL
{ cube2.sql() }
measures:
- name: total_transactions
sql: transaction_id
type: count
- name: total_amount
sql: amount
type: sum
- name: total_payout
sql: payout
type: sum
dimensions:
- name: user_id
sql: user_id
type: string
- name: currency
sql: currency
type: string
- name: at
sql: at
type: time
pre_aggregations:
- name: cube_total_rollup_lambda
type: rollup_lambda
union_with_source_data: true
rollups:
- CUBE.cube_total_rollup
- name: cube_total_rollup
type: rollup
measures:
- cube_total.total_transactions
- cube_total.total_amount
- cube_total.total_payout
dimensions:
- cube_total.user_id
- cube_total.currency
indexes:
- name: user_rollup_user_id_index
columns:
- cube_total.user_id
time_dimension: cube_total.at
granularity: quarter
external: true
partition_granularity: quarter
refresh_key:
every: 1 day
How can I make streams working with ClickHouse or switch query handler in Cube?
Already tried to turn on in Clickhouse session settings send_progress_in_http_headers assuming Cube uses custom http header X-ClickHouse-Progress (yes, couldn't find any evidences on it).
P.S Query fails after 2-3 seconds with that error, so it is definitely not await timeout.
Cubestore backend is S3