Issue: In Grafana Tempo, the Breakdown Panel Shows “File Not Found” Error #708
Description
Environment:
- Image: grafana/otel-lgtm:latest
- Running on Kubernetes
- Storage: Local file system (NFS)
What happens?
On Grafana’s Explore page, when I run any TraceQL query in the Breakdown panel, I always get a “file not found” error.
The same queries work fine in the Traces panel. ✅
Example error:
Query error: failed to execute TraceQL query: {resource.service.name != nil} | rate() by(resource.service.name)
Status: 500 Internal Server Error
Body: error opening file /data/tempo/generator/traces/single-tenant/[UUID]+single-tenant+vParquet4/0000000001: no such file or directory
Why does this happen?
After checking a bit, I noticed:
Tempo, with default LGTM settings, creates a new block every 1 second.
Within minutes, more than 10 blocks are created.
Most of these blocks are empty folders — only one of them has real parquet files.
The Traces panel works because it only reads from the index ✅
The Breakdown panel tries to scan all blocks, so it fails on the empty ones ❌