clickhouse routing for client hosted data plane

[Rainhunter13]

Note

Routes SQL queries and span writes to a workspace-specific data plane when in HYBRID mode, authenticated via short-lived JWTs; adds workspace routing metadata and DB migration.

• Routing based on deployment mode:
  • Add DeploymentMode enum and Workspace model with deployment_mode and optional data_plane_url in db/projects.rs.
  • New get_workspace_by_project_id to fetch workspace for routing.
• SQL API:
  • execute_sql_query now routes:
    • CLOUD/SELF_HOST → local ClickHouse (RO client).
    • HYBRID → new execute_sql_query_on_data_plane via HTTP POST {data_plane_url}/clickhouse/read with JWT.
  • Extracted validate_query helper; added request struct DataPlaneReadRequest.
• Traces consumer (writes):
  • On HYBRID, send spans to {data_plane_url}/clickhouse/write with JWT (DataPlaneWriteRequest); otherwise write to local ClickHouse.
  • Thread/app wiring passes shared reqwest::Client into workers.
• Utils (JWT):
  • New RS256 JWT generation (generate_data_plane_jwt) with cached tokens per workspace (moka), using DATA_PLANE_PRIVATE_KEY.
• Server wiring:
  • Instantiate and inject a shared reqwest::Client into HTTP handlers and consumer workers.
• Database migration:
  • Add workspaces.deployment_mode TEXT DEFAULT 'CLOUD' and workspaces.data_plane_url.
• Dependencies:
  • Add jsonwebtoken; lockfile updates for JWT-related crates.

^{Written by Cursor Bugbot for commit 45baa37. This will update automatically on new commits. Configure here.}

---

Important

Adds ClickHouse query routing based on deployment mode with JWT authentication for data plane communication.

• Behavior:
  • Adds execute_sql_query_on_data_plane() in sql/mod.rs for routing SQL queries to data plane in HYBRID mode.
  • Updates execute_sql_query() in api/v1/sql.rs to handle different deployment modes (CLOUD, SELF_HOST, HYBRID).
  • Implements JWT generation for data plane authentication in utils/mod.rs.
• Database:
  • Adds deployment_mode and data_plane_url columns to workspaces table in 0062_deployment_mode.sql.
  • Introduces DeploymentMode enum and get_workspace_by_project_id() in db/projects.rs.
• Dependencies:
  • Adds jsonwebtoken to Cargo.toml and Cargo.lock for JWT support.
• Misc:
  • Updates main.rs to initialize HTTP client for data plane communication.
  • Modifies traces/consumer.rs to route span data to data plane in HYBRID mode.

^{This description was created by for 45baa37. You can customize this summary. It will automatically update as commits are pushed.}

[ellipsis-dev]

Important

Looks good to me! 👍

Reviewed everything up to 45baa37 in 1 minute and 25 seconds. Click for details.

• Reviewed 826 lines of code in 9 files
• Skipped 0 files when reviewing.
• Skipped posting 6 draft comments. View those below.
• Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.

1. app-server/src/sql/mod.rs:46

• Draft comment:
  Consider precomputing the regex string for ERROR_END_REGEX instead of using format! inside the LazyLock closure. While it works, precomputing may improve clarity and avoid potential lifetime confusion.
• Reason this comment was not posted:
  Comment was not on a location in the diff, so it can't be submitted as a review comment.

2. app-server/src/main.rs:1040

• Draft comment:
  The main function spawns several threads and later joins them. Consider adding more robust error handling for thread panics and documenting the design intent to ensure that blocking operations via block_on in spawned threads are safe.
• Reason this comment was not posted:
  Comment was not on a location in the diff, so it can't be submitted as a review comment.

3. app-server/src/traces/consumer.rs:191

• Draft comment:
  Mixing Rayon’s parallel iterator for heavy CPU-bound work with async tasks is a good strategy, but ensure that this parallel processing does not starve your Tokio runtime. Consider verifying that the Rayon thread pool is appropriately tuned or use a dedicated CPU-bound thread pool if needed.
• Reason this comment was not posted:
  Comment was not on a location in the diff, so it can't be submitted as a review comment.

4. app-server/src/utils/mod.rs:112

• Draft comment:
  In the sanitize_string function, you filter out control characters including null and invalid UTF-8. Double–check that this filtering does not unintentionally remove characters that might be valid in your application context.
• Reason this comment was not posted:
  Confidence changes required: 50% <= threshold 50% None

5. frontend/lib/db/migrations/0062_deployment_mode.sql:1

• Draft comment:
  The migration adds new columns for deployment mode and data plane URL. Ensure that these schema changes are backward–compatible and consider whether an index on deployment_mode might improve query performance if used frequently.
• Reason this comment was not posted:
  Confidence changes required: 50% <= threshold 50% None

6. app-server/src/traces/consumer.rs:82

• Draft comment:
  Typo detected: "conneciton" should be corrected to "connection".
• Reason this comment was not posted:
  Comment was on unchanged code.

Workflow ID: wflow_RY9jTcYS91uTy27t

^{You can customize by changing your verbosity settings, reacting with 👍 or 👎, replying to comments, or adding code review rules.}

[cursor]

Bug: CHSpan UUID serialization incompatible with JSON format

The DataPlaneWriteRequest serializes CHSpan data to JSON via reqwest::Client::json(). However, CHSpan uses #[serde(with = "clickhouse::serde::uuid")] for UUID fields, which is designed for ClickHouse's binary protocol, not JSON serialization. When serialized to JSON, UUID fields (span_id, parent_span_id, project_id, trace_id) will use the ClickHouse-specific format instead of standard JSON UUID strings, likely causing the data plane to fail parsing or storing corrupted data.