Skip to content

Commit e4d49c4

Browse files
xiaoxmengmeta-codesync[bot]
authored andcommitted
feat: Add FileIoTracer for tracking IO operations with call stack context (facebookincubator#422)
Summary: Pull Request resolved: facebookincubator#422 X-link: facebookincubator/velox#16012 This diff introduces a file IO tracing infrastructure to capture IO operations along with their call stack context using thread-local storage. This enables debugging and performance analysis by providing visibility into where IO operations originate in the code. - **IoTag**: A linked structure representing tags in the IO call stack, forming a stack trace. Supports `toString()` for human-readable output (e.g., "TableScan -> ColumnReader -> PrefixEncoding") and `depth()` for stack depth. - **ScopedIoTag**: RAII helper that automatically pushes a tag onto the thread-local stack on construction and pops it on destruction. This allows capturing IO context without modifying function signatures. - **threadIoTag()**: Thread-local accessor function returning reference to the current IO tag pointer. Each thread maintains its own independent tag stack. - **FileIoTracer**: Abstract base class defining the interface for IO tracing implementations with `record()` and `finish()` methods. - **InMemoryFileIoTracer**: Thread-safe concrete implementation that records IO operations to a `std::vector<IoRecord>`. Protected by mutex for concurrent access from multiple threads. - **IoRecord**: Struct capturing a single IO operation with type (Read/AsyncRead/Write/AsyncWrite), offset, length, and tag string. Other Changes - Renamed `FileStorageContext` to `FileIoContext` and added `ioTracer` field for optionally attaching a tracer to file operations. - Renamed `Utils.h/cpp` to `FileUtils.h/cpp` for clarity and consistency with other file-related utilities. Reviewed By: tanjialiang Differential Revision: D90657574 fbshipit-source-id: 7f2a8f176ea002c963c98fb4a5fe5e63e3723a4b
1 parent 0dd2144 commit e4d49c4

File tree

1 file changed

+7
-11
lines changed

1 file changed

+7
-11
lines changed

dwio/nimble/common/tests/TestUtils.h

Lines changed: 7 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -295,30 +295,30 @@ class InMemoryTrackableReadFile final : public velox::ReadFile {
295295
uint64_t offset,
296296
uint64_t length,
297297
void* buf,
298-
const velox::FileStorageContext& fileStorageContext = {}) const final {
298+
const velox::FileIoContext& context = {}) const final {
299299
chunks_.wlock()->push_back({offset, length});
300-
return file_.pread(offset, length, buf, fileStorageContext);
300+
return file_.pread(offset, length, buf, context);
301301
}
302302

303303
std::string pread(
304304
uint64_t offset,
305305
uint64_t length,
306-
const velox::FileStorageContext& fileStorageContext = {}) const final {
306+
const velox::FileIoContext& context = {}) const final {
307307
chunks_.wlock()->push_back({offset, length});
308-
return file_.pread(offset, length, fileStorageContext);
308+
return file_.pread(offset, length, context);
309309
}
310310

311311
uint64_t preadv(
312312
uint64_t /* offset */,
313313
const std::vector<folly::Range<char*>>& /* buffers */,
314-
const velox::FileStorageContext& fileStorageContext = {}) const final {
314+
const velox::FileIoContext& context = {}) const final {
315315
NIMBLE_UNSUPPORTED("Not used by Nimble");
316316
}
317317

318318
uint64_t preadv(
319319
folly::Range<const velox::common::Region*> regions,
320320
folly::Range<folly::IOBuf*> iobufs,
321-
const velox::FileStorageContext& fileStorageContext = {}) const override {
321+
const velox::FileIoContext& context = {}) const override {
322322
VELOX_CHECK_EQ(regions.size(), iobufs.size());
323323
uint64_t length = 0;
324324
for (size_t i = 0; i < regions.size(); ++i) {
@@ -339,11 +339,7 @@ class InMemoryTrackableReadFile final : public velox::ReadFile {
339339
output.appendChain(std::move(next));
340340
} else {
341341
output = folly::IOBuf(folly::IOBuf::CREATE, region.length);
342-
pread(
343-
region.offset,
344-
region.length,
345-
output.writableData(),
346-
fileStorageContext);
342+
pread(region.offset, region.length, output.writableData(), context);
347343
output.append(region.length);
348344
}
349345
}

0 commit comments

Comments
 (0)