docs: Updates (#7277)

* Minimal permissions for SLS in Preset

* Fix cube_dbt code examples

* Add a note on unsafe string escaping

* Clarify building custom Docker images

* Garbage collection in Cube Store

* Fix case dimension

Author: igorlukanin

docs/docs-new/pages/guides/dbt.mdx

@@ -176,7 +176,7 @@ This is particularly useful if you keep your data marts under
 from cube_dbt import Dbt
 
 manifest_url = 'https://cube-dbt-integration.s3.amazonaws.com/manifest.json'
-dbt = Dbt.from_url(manifest_url).filter(paths=['models/marts/'])
+dbt = Dbt.from_url(manifest_url).filter(paths=['marts/'])
 ```
 
 You can also load only models with certain tags. This is useful if you'd
@@ -211,7 +211,7 @@ from cube import TemplateContext
 from cube_dbt import Dbt
 
 manifest_url = 'https://cube-dbt-integration.s3.amazonaws.com/manifest.json'
-dbt = Dbt.from_url(manifest_url).filter(paths=['models/marts/'])
+dbt = Dbt.from_url(manifest_url).filter(paths=['marts/'])
 
 template = TemplateContext()
 

docs/docs-new/pages/product/caching/running-in-production.mdx

@@ -216,18 +216,24 @@ using Cube Cloud.
 
 ## Storage
 
-<WarningBox>
+Cube Store cluster uses both persistent and scratch storage.
 
-Cube Store can only use one type of remote storage at runtime.
-
-</WarningBox>
+### Persistent storage
 
 Cube Store makes use of a separate storage layer for storing metadata as well as
-for persisting pre-aggregations as Parquet files. Cube Store [can be configured
-to use either AWS S3 or Google Cloud Storage][ref-config-env]. If desired, local
-path on the server can also be used in case all Cube Store cluster nodes are
+for persisting pre-aggregations as Parquet files.
+
+Cube Store [can be configured][ref-config-env] to use either AWS S3 or
+Google Cloud Storage (GCS) as persistent storage. If desired, local path on
+the server can also be used in case all Cube Store cluster nodes are
 co-located on a single machine.
 
+<InfoBox>
+
+Cube Store can only use one type of remote storage at runtime.
+
+</InfoBox>
+
 <WarningBox>
 
 Cube Store requires strong consistency guarantees from underlying distributed
@@ -242,6 +248,7 @@ A simplified example using AWS S3 might look like:
 
 ```yaml
 version: "2.2"
+
 services:
   cubestore_router:
     image: cubejs/cubestore:latest
@@ -253,6 +260,7 @@ services:
       - CUBESTORE_S3_REGION=<BUCKET_REGION_IN_S3>
       - CUBESTORE_AWS_ACCESS_KEY_ID=<AWS_ACCESS_KEY_ID>
       - CUBESTORE_AWS_SECRET_ACCESS_KEY=<AWS_SECRET_ACCESS_KEY>
+
   cubestore_worker_1:
     image: cubejs/cubestore:latest
     environment:
@@ -268,12 +276,16 @@ services:
       - cubestore_router
 ```
 
-### Local Storage
+Note that you can’t use the same bucket as an export bucket and persistent
+storage for Cube Store. It's recommended to use two separate buckets. 
+
+### Scratch storage
 
-Separately from remote storage, Cube Store requires local scratch space to warm
-up partitions by downloading Parquet files before querying them. By default,
-this directory should be mounted to `.cubestore/data` dir inside contained and
-can be configured by [CUBESTORE_DATA_DIR][ref-config-env] environment variable.
+Separately from persistent storage, Cube Store requires local scratch space
+to warm up partitions by downloading Parquet files before querying them.
+
+By default, this folder should be mounted to `.cubestore/data` inside the
+container and can be configured by `CUBESTORE_DATA_DIR` environment variable.
 It is advised to use local SSDs for this scratch space to maximize querying
 performance.
 
@@ -293,6 +305,16 @@ default.
 
 </WarningBox>
 
+### Garbage collection
+
+Cleanup isn’t done in export buckets; however, it's done in the persistent
+storage of Cube Store. The default time-to-live (TTL) for orphaned
+pre-aggregation tables is one day.
+
+Refresh worker should be able to finish pre-aggregation refresh before
+garbage collection starts. It means that all pre-aggregation partitions
+should be built before any tables are removed.
+
 ## Security
 
 Cube Store currently does not have any in-built authentication mechanisms. For

docs/docs-new/pages/product/configuration.mdx

@@ -120,19 +120,36 @@ Docker container.
 You can edit the configuration file by going into <Btn>Development Mode</Btn>
 and navigating to the <Btn>Data Model</Btn> page.
 
-## Runtime and dependencies
+## Runtimes and dependencies
 
-As of v0.34, Cube uses Node.js v16.20 to execute JavaScript-based
-configuration and Python v3.9 for Python-based configuration.
+As of v0.34, Cube uses Python v3.9 and Node.js v16.20 as runtime environments
+for the code of configuration and [dynamic data models][ref-dynamic-data-models].
 
-If you have dependencies in `requirements.txt` file, make sure to install
-them by running `pip install -r requirements.txt` inside the container.
+It's recommended to use `requirements.txt` and `package.json` files to
+specify dependencies for your Python and JavaScript code, respectively.
 
-If you have dependencies in `package.json` file, make sure to install them
-by running `yarn` and mount the `node_modules` folder under `/cube/conf`
-in the Docker container.
+### Cube Core
+
+If you have specified Python packages in the `requirements.txt` file,
+make sure to install them by running `pip install -r requirements.txt`
+inside the Docker container.
+
+If you have specified npm packages in the `package.json` file, make sure
+to install them by running `npm install` inside the Docker container.
+Alternatively, you can run `npm install` on your local machine and mount
+the `node_modules` folder under `/cube/conf` in the Docker container;
+however, if your dependencies contain native extensions, they might not work
+when provided this way.
+
+To automate the installation of dependencies, build and use a [custom
+Docker image][ref-custom-docker-image].
+
+### Cube Cloud
+
+Cube Cloud automatically installs dependencies from `requirements.txt` and
+`package.json` files.
 
-## Development Mode
+## Development mode
 
 Cube can be run in an insecure, development mode by setting the
 `CUBEJS_DEV_MODE` environment variable to `true`. Putting Cube in development
@@ -163,5 +180,7 @@ mode does the following:
 [link-env-vars]: /reference/configuration/environment-variables
 [link-scheduled-refresh]:
   /reference/data-model/pre-aggregations#scheduled_refresh
+[ref-dynamic-data-models]: /product/data-modeling/dynamic
+[ref-custom-docker-image]: /product/deployment/core#extend-the-docker-image
 
 [link-docker-env-vars]: https://docs.docker.com/compose/environment-variables/set-environment-variables/

docs/docs-new/pages/product/data-modeling/dynamic/_meta.js

@@ -2,5 +2,5 @@ module.exports = {
   "jinja": "Dynamic data models with Jinja and Python",
   "javascript": "Dynamic data models with JavaScript",
   "code-reusability-export-and-import": "Export and import",
-  "schema-execution-environment": "Execution Environment (JS models)"
+  "schema-execution-environment": "Execution environment (JavaScript models)"
 }

docs/docs-new/pages/product/data-modeling/dynamic/jinja.mdx

@@ -112,6 +112,34 @@ cubes:
       FROM app_data.payments
 ```
 
+### Escaping unsafe strings
+
+[Auto-escaping][jinja-docs-autoescaping] of unsafe string values in Jinja
+templates is enabled by default. It means that any strings coming from Python
+might get wrapped in quotes, potentially breaking YAML syntax.
+
+You can work around that by using the [`safe` Jinja
+filter][jinja-docs-filters-safe] with such string values:
+
+```yaml
+cubes:
+  - name: my_cube
+    description: {{ get_unsafe_string() | safe }}
+```
+
+Alternatively, you can wrap unsafe strings into instances of the following
+class in your Python code, effectively marking them as safe. This is
+particularly useful for library code, e.g., similar to the
+[`cube_dbt`][ref-cube-dbt] package.
+
+```python
+class SafeString(str):
+  is_safe: bool
+
+  def __init__(self, v: str):
+    self.is_safe = True
+```
+
 ## Python
 
 You can declare and invoke Python functions from within a Jinja template. This
@@ -201,3 +229,6 @@ cubes:
 [jinja-docs-for-loop]: https://jinja.palletsprojects.com/en/3.1.x/templates/#for
 [jinja-docs-macros]:
   https://jinja.palletsprojects.com/en/3.1.x/templates/#macros
+[jinja-docs-autoescaping]: https://jinja.palletsprojects.com/en/3.1.x/api/#autoescaping
+[jinja-docs-filters-safe]: https://jinja.palletsprojects.com/en/3.1.x/templates/#jinja-filters.safe
+[ref-cube-dbt]: /reference/python/cube_dbt

docs/docs-new/pages/product/deployment/core.mdx

@@ -280,10 +280,12 @@ services:
 
 ## Extend the Docker image
 
-If you need to use npm packages with native extensions inside [the `cube.js`
-configuration file][ref-config-js], you'll need to build your own Docker image.
-You can do this by first creating a `Dockerfile` and a corresponding
-`.dockerignore`:
+If you need to use dependencies (i.e., Python or npm packages) with native
+extensions inside [configuration files][ref-config-files] or [dynamic data
+models][ref-dynamic-data-models], build a custom Docker image.
+
+You can do this by creating a `Dockerfile` and a corresponding
+`.dockerignore` file:
 
 ```bash{promptUser: user}
 touch Dockerfile
@@ -296,23 +298,26 @@ Add this to the `Dockerfile`:
 FROM cubejs/cube:latest
 
 COPY . .
+RUN apt update && apt install -y pip
+RUN pip install -r requirements.txt
 RUN npm install
 ```
 
 And this to the `.dockerignore`:
 
 ```gitignore
-node_modules
-npm-debug.log
 schema
+cube.py
 cube.js
 .env
+node_modules
+npm-debug.log
 ```
 
 Then start the build process by running the following command:
 
 ```bash{promptUser: user}
-docker build -t <YOUR-USERNAME>/cubejs-custom-build .
+docker build -t <YOUR-USERNAME>/cube-custom-image .
 ```
 
 Finally, update your `docker-compose.yml` to use your newly-built image:
@@ -322,31 +327,41 @@ version: "2.2"
 
 services:
   cube_api:
-    image: <YOUR-USERNAME>/cubejs-custom-build
+    image: <YOUR-USERNAME>/cube-custom-image
     ports:
       - 4000:4000
     environment:
-      - CUBEJS_DB_TYPE=bigquery
-      - CUBEJS_DB_BQ_PROJECT_ID=cubejs-bq-cluster
-      - CUBEJS_DB_BQ_CREDENTIALS=<BQ-KEY>
-      - CUBEJS_DB_EXPORT_BUCKET=cubestore
-      - CUBEJS_CUBESTORE_HOST=cubestore_router
       - CUBEJS_API_SECRET=secret
+      # Other environment variables
     volumes:
       - .:/cube/conf
-      # Prevent dev dependencies leaking
-      - .empty:/cube/conf/node_modules/@cubejs-backend/
     depends_on:
       - cubestore_router
       - cube_refresh_worker
+      # Other container dependencies
+```
+
+Note that you shoudn't mount the whole current folder (`.:/cube/conf`)
+if you have dependencies in `package.json`. Doing so would effectively
+hide the `node_modules` folder inside the container, where dependency files
+installed with `npm install` reside, and result in errors like this:
+`Error: Cannot find module 'my_dependency'`. In that case, mount individual files:
+
+```yaml
+    # ...
+    volumes:
+      - ./schema:/cube/conf/schema
+      - ./cube.js:/cube/conf/cube.js
+      # Other necessary files
 ```
 
 [medium-letsencrypt-nginx]:
   https://pentacent.medium.com/nginx-and-lets-encrypt-with-docker-in-less-than-5-minutes-b4b8a60d3a71
 [link-cubejs-docker]: https://hub.docker.com/r/cubejs/cube
 [link-docker-app]: https://www.docker.com/products/docker-app
 [link-nginx]: https://www.nginx.com/
-[ref-config-js]: /reference/configuration/config
+[ref-config-files]: /product/configuration#cubepy-and-cubejs-files
+[ref-dynamic-data-models]: /product/data-modeling/dynamic
 [ref-config-queryrewrite]: /reference/configuration/config#queryrewrite
 [ref-config-sched-ref-ctx]:
   /reference/configuration/config#scheduledrefreshcontexts

docs/docs-new/pages/product/workspace/semantic-layer-sync.mdx

@@ -311,10 +311,13 @@ module.exports = {
 ### Preset
 
 Data model is synchronized via [Preset API][preset-api] which uses API keys for
-authentication. You can generate a new API key in your [user
+authentication.
+
+You can generate a new API key in your [user
 settings][preset-user-settings] in Preset to obtain an `api_token` and an
 `api_secret`. You can also copy a `workspace_url` at any page of your Preset
-workspace.
+workspace. Note that your use should have the [workspace
+role][preset-user-roles] of `Workspace Admin`.
 
 Example confguration for Preset:
 
@@ -538,6 +541,7 @@ and clicking <Btn>Show History</Btn> next to a relevant sync.
 [ref-dev-mode]: /product/workspace/dev-mode
 [preset-api]: https://api-docs.preset.io
 [preset-user-settings]: https://manage.app.preset.io/app/user
+[preset-user-roles]: https://docs.preset.io/docs/role-based-access-security-rbac#change-user-access-to-a-workspace
 [superset-api]: https://superset.apache.org/docs/api/
 [metabase-api]:
   https://www.metabase.com/learn/administration/metabase-api#authenticate-your-requests-with-a-session-token