Skip iterating DISI when reading metric values

[dnhatn]

Avoids iterating the DISI when reading a sparse metric field if its nulls were already filtered by a WHERE clause (e.g., metrics != NULL). Iterating the DISI on a sparse field can be expensive. From my local benchmark, the query time of

TS my* 
  | WHERE `metrics.system.memory.utilization` IS NOT NULL 
     AND @timestamp >= \"2025-07-25T14:55:59.000Z\" 
      AND @timestamp <= \"2025-07-25T16:25:59.000Z\" 
  | STATS AVG(AVG_OVER_TIME(`metrics.system.memory.utilization`)) BY host.name, BUCKET(@timestamp, 1h) 
  | LIMIT 10000"

decreased from 35ms to 27ms (20%).

Before:

After:

[kkrik-es]

Nice, I'm glad my draft has some overlap :)

[martijnvg]

This looks good. I left minor comments and we need a test for tryReadDoubles() in ES819TSDBDocValuesFormatTests

[dnhatn]

@martijnvg @kkrik-es I've updated this with Kostas's PR. I think it looks better. Can you take a look? Thanks!

[elasticsearchmachine]

Pinging @elastic/es-storage-engine (Team:StorageEngine)

[elasticsearchmachine]

Hi @dnhatn, I've created a changelog YAML for you.

[martijnvg]

LGTM, thanks Nhat!

[kkrik-es]

Nice and clean!

[dnhatn]

@martijnvg @kkrik-es Thank you!