Skip to content

Conversation

@JackieTien97
Copy link
Contributor

  1. Add cache hit situation and actual io size for BloomFilter, TimeSeriesMetadata(including ChunkMetadatList) and Chunk
  2. print them in explain analyze verbose, like
- BloomFilter 相关(用于判断某个序列在tsfile中是否存在,存储于tsfile尾部)
  - loadBloomFilterFromCacheCount: 命中BloomFilterCache的次数
  - loadBloomFilterFromDiskCount: 从磁盘中读取BloomFilter的次数
  - loadBloomFilterActualIOSize: 从磁盘中读取BloomFilter时耗费的磁盘IO(单位为bytes)
  - loadBloomFilterTime: 读取BloomFilter + 计算序列是否存在的总耗时(单位为ms)
- TimeSeriesMetadata相关
  - loadTimeSeriesMetadataFromCacheCount: 命中TimeSeriesMetadataCache的次数,注意这里对于对齐设备来讲,每个分量(包括time列,都会去单独请求Cache,所以对于对齐设备来讲,(loadTimeSeriesMetadataFromCacheCount + loadTimeSeriesMetadataFromDiskCount)= tsfile number * subSensor number(包括time列)
  - loadTimeSeriesMetadataFromDiskCount: 从磁盘中读取TimeSeriesMetadata的次数,一次读取会把该设备下查询所有涉及到的分量都读出来缓存在TimeSeriesMetadataCache中
  - loadTimeSeriesMetadataActualIOSize: 从磁盘中读取TimeSeriesMetadata时耗费的磁盘IO(单位为bytes)
- Mods 文件相关
  - TimeSeriesMetadataModificationTime: 读取mods文件花费的时间
- Chunk 相关
  - loadChunkFromCacheCount: 命中ChunkCache的次数,注意这里对于对齐设备来讲,每个分量(包括time列,都会去单独请求Cache,所以对于对齐设备来讲,(loadChunkFromCacheCount + loadChunkFromDiskCount)= tsfile number * subSensor number(包括time列)* avg chunk number in each tsfile
  - loadChunkFromDiskCount: 从磁盘中读取Chunk的次数,一次只会读取一个分量,并缓存在ChunkCache中
  - loadChunkActualIOSize: 从磁盘中读取Chunk时耗费的磁盘IO(单位为bytes)
  1. Also put them in IoTDB Metrics

@sonarqubecloud
Copy link

@codecov
Copy link

codecov bot commented Nov 13, 2024

Codecov Report

Attention: Patch coverage is 39.93399% with 182 lines in your changes missing coverage. Please review.

Project coverage is 39.71%. Comparing base (32e701e) to head (cd47428).
Report is 3 commits behind head on master.

Files with missing lines Patch % Lines
...db/queryengine/metric/SeriesScanCostMetricSet.java 10.60% 118 Missing ⚠️
...ne/execution/fragment/FragmentInstanceContext.java 0.00% 16 Missing ⚠️
...e/statistics/FragmentInstanceStatisticsDrawer.java 0.00% 14 Missing ⚠️
...ueryengine/execution/fragment/QueryStatistics.java 64.51% 11 Missing ⚠️
.../storageengine/buffer/TimeSeriesMetadataCache.java 80.55% 7 Missing ⚠️
...otdb/db/storageengine/buffer/BloomFilterCache.java 76.00% 6 Missing ⚠️
...ache/iotdb/db/storageengine/buffer/ChunkCache.java 85.00% 6 Missing ⚠️
...tdb/commons/exception/IoTDBIORuntimeException.java 0.00% 3 Missing ⚠️
...ine/dataregion/read/control/FileReaderManager.java 66.66% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #14075      +/-   ##
============================================
- Coverage     39.72%   39.71%   -0.02%     
  Complexity       71       71              
============================================
  Files          4216     4216              
  Lines        267840   267909      +69     
  Branches      32488    32453      -35     
============================================
- Hits         106400   106393       -7     
- Misses       161440   161516      +76     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@JackieTien97 JackieTien97 merged commit d637770 into master Nov 13, 2024
36 of 38 checks passed
@JackieTien97 JackieTien97 deleted the QueryIO branch November 13, 2024 12:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant