Skip to content

Commit dfb6a76

Browse files
authored
docs: Streaming mode in the SQL API (#8593)
1 parent 361f0f0 commit dfb6a76

File tree

3 files changed

+31
-4
lines changed

3 files changed

+31
-4
lines changed

docs/pages/product/apis-integrations/queries.mdx

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -33,13 +33,17 @@ from default ordering by passing an empty array (`[]`) object as `order`.
3333

3434
### Row limit
3535

36-
By default, [any query](#query-types) will return no more than 10,000 rows
37-
in the result set. It serves as a safeguard against data scraping and
38-
denial-of-service (DoS) attacks if Cube is exposed to untrusted environments.
36+
By default, [any query](#query-types) that does not explicitly specify the row
37+
limit, will return no more than 10,000 rows in the result set. You can use the
38+
`CUBEJS_DB_QUERY_DEFAULT_LIMIT` environment variable to override it. This serves
39+
as a safeguard against data scraping and denial-of-service (DoS) attacks if Cube
40+
is exposed to untrusted environments.
3941

4042
The maximum allowed limit is 50,000 rows. You can use the `CUBEJS_DB_QUERY_LIMIT`
4143
environment variable to override it. You can also implement
4244
[pagination][ref-pagination-recipe] to fetch more rows than the maximum limit.
45+
Finally, if you're using the [SQL API][ref-sql-api], you can enable the
46+
[streaming mode][ref-sql-api-streaming] to ignore the maximum row limit.
4347

4448
### Time zone
4549

@@ -394,3 +398,4 @@ Additionally, note that ungrouped queries have additional requirements for
394398
[ref-query-rewrite]: /reference/configuration/config#query_rewrite
395399
[ref-ref-sql-api]: /reference/sql-api
396400
[blog-compare-date-range]: https://cube.dev/blog/comparing-data-over-different-time-periods
401+
[ref-sql-api-streaming]: /product/apis-integrations/sql-api#streaming

docs/pages/product/apis-integrations/sql-api.mdx

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -211,6 +211,18 @@ error: `Error during rewrite: Can't find rewrite due to 10002 AST node limit rea
211211
Use the following environment variables to allocate more resources for query planning:
212212
`CUBESQL_REWRITE_MAX_NODES`, `CUBESQL_REWRITE_MAX_ITERATIONS`, `CUBESQL_REWRITE_TIMEOUT`.
213213

214+
## Streaming
215+
216+
By default, query results are loaded in a single batch. However, a more effective
217+
*streaming mode* can be used for large result sets. To enable it, set the
218+
`CUBESQL_STREAM_MODE` environment variable to `true`.
219+
220+
<InfoBox>
221+
222+
When the streaming mode is enabled, the maximum [row limit][ref-queries-row-limit]
223+
does not apply to SQL API queries. They can return an unlimited number of rows.
224+
225+
</InfoBox>
214226

215227
[ref-sql-api-auth]: /product/apis-integrations/sql-api/security
216228
[ref-config-checksqlauth]: /reference/configuration/config#checksqlauth
@@ -241,4 +253,5 @@ Use the following environment variables to allocate more resources for query pla
241253
[link-postgres-information-schema]: https://www.postgresql.org/docs/16/information-schema.html
242254
[link-postgres-backslash-commands]: https://www.postgresql.org/docs/current/app-psql.html#APP-PSQL-META-COMMANDS
243255
[postgres-protocol]: https://www.postgresql.org/docs/current/protocol.html
244-
[cube-bi-use-case]: https://cube.dev/use-cases/connected-bi
256+
[cube-bi-use-case]: https://cube.dev/use-cases/connected-bi
257+
[ref-queries-row-limit]: /product/apis-integrations/queries#row-limit

docs/pages/reference/configuration/environment-variables.mdx

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -676,6 +676,15 @@ The username used to connect to the database.
676676
| ------------------------- | ---------------------- | --------------------- |
677677
| A valid database username | N/A | N/A |
678678

679+
## `CUBEJS_DB_QUERY_DEFAULT_LIMIT`
680+
681+
The default [row limit][ref-row-limit] in the result set if it's not explicitly
682+
specified in an API query.
683+
684+
| Possible Values | Default in Development | Default in Production |
685+
| ------------------------- | ---------------------- | --------------------- |
686+
| A positive integer number | `10000` | `10000` |
687+
679688
## `CUBEJS_DB_QUERY_LIMIT`
680689

681690
The maximum [row limit][ref-row-limit] in the result set.

0 commit comments

Comments
 (0)