Search before asking

• I had searched in the issues and found no similar feature requirement.

Description

Design a comprehensive schema to capture all essential tracing constructs (traces, spans, tags) while optimizing for storage efficiency and query performance.

Key Features:

• Sparse Primary Index: Implement a sparse index for the traceID primary key, with rows ordered by traceID to enable fast point queries and range scans.
• Query Column Files: Store each queryable column (e.g., serviceName, operationName, status) in separate files. These files must support skipping indexes (min-max, bloom filter) to accelerate filtering. Rows in these files are ordered by traceID-spanID to maintain alignment with span data.
• Span Data File: Store span data (e.g., start/end timestamps, tags, logs) in a dedicated file, ordered by traceID-spanID. This ordering ensures that spans from the same trace are grouped together for efficient retrieval.
• Merge Optimization: For non-overlapping span files during merging (e.g., no overlapping traceID-spanID ranges), support direct concatenation without scanning file content to minimize merge time.

Use case

No response

Related issues

No response

Are you willing to submit a pull request to implement this on your own?

• Yes I am willing to submit a pull request on my own!

Code of Conduct

• I agree to follow this project's Code of Conduct