Skip to content

[clickhouse] Setup creation of dependencies table in ClickHouse storage#8329

Open
mahadzaryab1 wants to merge 7 commits intojaegertracing:mainfrom
mahadzaryab1:ch-deps-table
Open

[clickhouse] Setup creation of dependencies table in ClickHouse storage#8329
mahadzaryab1 wants to merge 7 commits intojaegertracing:mainfrom
mahadzaryab1:ch-deps-table

Conversation

@mahadzaryab1
Copy link
Copy Markdown
Collaborator

Which problem is this PR solving?

Description of the changes

  • Sets up a new dependencies table for ClickHouse as part of an effort to implement dependency storage in ClickHouse.

How was this change tested?

  • CI / Unit Tests

Checklist

AI Usage in this PR (choose one)

See AI Usage Policy.

  • None: No AI tools were used in creating this PR
  • Light: AI provided minor assistance (formatting, simple suggestions)
  • Moderate: AI helped with code generation or debugging specific parts
  • Heavy: AI generated most or all of the code changes

Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
@mahadzaryab1 mahadzaryab1 requested a review from yurishkuro April 9, 2026 04:46
@mahadzaryab1 mahadzaryab1 requested a review from a team as a code owner April 9, 2026 04:46
@mahadzaryab1 mahadzaryab1 added the changelog:experimental Change to an experimental part of the code label Apr 9, 2026
Copilot AI review requested due to automatic review settings April 9, 2026 04:46
@dosubot dosubot bot added the area/storage label Apr 9, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds initial ClickHouse schema support for dependency storage (towards implementing the v2 depstore.Reader on ClickHouse) by introducing a new dependencies table and wiring it into schema creation and purge flows.

Changes:

  • Add create_dependencies_table.sql and embed it as sql.CreateDependenciesTable.
  • Create the dependencies table during ClickHouse factory schema initialization.
  • Include dependencies table truncation in Factory.Purge() and extend unit tests for the new create/truncate paths.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File Description
internal/storage/v2/clickhouse/sql/queries.go Adds embedded schema query and a truncate query constant for the new dependencies table.
internal/storage/v2/clickhouse/sql/create_dependencies_table.sql Introduces the ClickHouse dependencies table DDL.
internal/storage/v2/clickhouse/factory.go Ensures the dependencies table is created when CreateSchema is enabled and purged via truncate.
internal/storage/v2/clickhouse/factory_test.go Adds negative-path unit tests for dependencies table creation and truncation failures.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

CREATE TABLE IF NOT EXISTS
dependencies (
timestamp DateTime64 (9),
dependencies String
Copy link

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new table schema uses a column named dependencies inside a table also named dependencies. This makes queries/readability harder (e.g., SELECT dependencies FROM dependencies) and can be confusing when adding readers/writers later. Consider renaming the column to something more specific (e.g., dependencies_json, links, or payload) to avoid the table/column name collision.

Suggested change
dependencies String
dependencies_json String

Copilot uses AI. Check for mistakes.
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 9, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.59%. Comparing base (a77df70) to head (1dc2339).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff             @@
##             main    #8329       +/-   ##
===========================================
+ Coverage   37.88%   95.59%   +57.71%     
===========================================
  Files         169      314      +145     
  Lines       10121    16457     +6336     
===========================================
+ Hits         3834    15732    +11898     
+ Misses       5893      571     -5322     
+ Partials      394      154      -240     
Flag Coverage Δ
badger_direct 9.32% <0.00%> (+0.29%) ⬆️
badger_e2e 1.08% <0.00%> (+0.04%) ⬆️
cassandra-4.x-direct-manual 13.66% <0.00%> (+0.46%) ⬆️
cassandra-4.x-e2e-auto 1.06% <0.00%> (+0.04%) ⬆️
cassandra-4.x-e2e-manual 1.06% <0.00%> (+0.04%) ⬆️
cassandra-5.x-direct-manual 13.66% <0.00%> (+0.46%) ⬆️
cassandra-5.x-e2e-auto 1.06% <0.00%> (+0.04%) ⬆️
cassandra-5.x-e2e-manual 1.06% <0.00%> (+0.04%) ⬆️
clickhouse 1.20% <0.00%> (+0.04%) ⬆️
elasticsearch-6.x-direct 17.50% <0.00%> (+0.74%) ⬆️
elasticsearch-7.x-direct 17.53% <0.00%> (+0.74%) ⬆️
elasticsearch-8.x-direct 17.69% <0.00%> (+0.75%) ⬆️
elasticsearch-8.x-e2e 1.08% <0.00%> (+0.04%) ⬆️
elasticsearch-9.x-e2e 1.08% <0.00%> (+0.04%) ⬆️
grpc_direct 8.11% <0.00%> (+0.34%) ⬆️
grpc_e2e 1.08% <0.00%> (+0.04%) ⬆️
kafka-3.x-v2 1.08% <0.00%> (+0.04%) ⬆️
memory_v2 1.08% <0.00%> (+0.04%) ⬆️
opensearch-1.x-direct 17.58% <0.00%> (+0.74%) ⬆️
opensearch-2.x-direct 17.58% <0.00%> (+0.74%) ⬆️
opensearch-2.x-e2e 1.08% <0.00%> (+0.04%) ⬆️
opensearch-3.x-e2e 1.08% <0.00%> (+0.04%) ⬆️
query 1.08% <0.00%> (+0.04%) ⬆️
tailsampling-processor 0.54% <0.00%> (+0.02%) ⬆️
unittests 94.23% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
Signed-off-by: Mahad Zaryab <mahadzaryab1@gmail.com>
Copilot AI review requested due to automatic review settings April 9, 2026 05:39
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +273 to +277
const TruncateSpans = `TRUNCATE TABLE IF EXISTS spans`

const TruncateServices = `TRUNCATE TABLE services`
const TruncateServices = `TRUNCATE TABLE IF EXISTS services`

const TruncateOperations = `TRUNCATE TABLE operations`
const TruncateOperations = `TRUNCATE TABLE IF EXISTS operations`
Copy link

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switching the TRUNCATE statements to TRUNCATE TABLE IF EXISTS ... changes Purge behavior: it will now succeed even when tables are missing (e.g., wrong DB/schema), which can mask misconfiguration and make tests/ops think data was purged when it wasn’t. Consider keeping TRUNCATE strict (no IF EXISTS) or explicitly validating required tables exist and returning a clear error if they don’t.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/storage changelog:experimental Change to an experimental part of the code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants