storage: avoid excessive walking of the auxiliary directory #158160
Conversation
|
It looks like your PR touches production code but doesn't add or edit any test code. Did you consider adding tests to your PR? 🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf. |
|
This change is |
This comment was marked as resolved.
This comment was marked as resolved.
github-actions
bot
added
the
o-AI-Review-Potential-Issue-Detected
RaduBerinde
left a comment
RaduBerinde
left a comment
There was a problem hiding this comment.
Reviewable status:
complete! 0 of 0 LGTMs obtained
pkg/storage/pebble.go line 812 at r1 (raw file):
auxiliarySize struct { mu syncutil.Mutex computedAt time.Time
[nit] crtime.Mono
pkg/storage/pebble.go line 1991 at r1 (raw file):
// Special-case: if the store-dir is configured using the root of some fs, // e.g. "/mnt/db", we might have special fs-created files like lost+found // that we can't read, so just ignore them rather than crashing.
[nit] s/crashing/erroring out
RaduBerinde
left a comment
RaduBerinde
left a comment
There was a problem hiding this comment.
Reviewable status:
Pebble incrementally accounts its disk usage and this accounting is used to cheaply service Capacity calls. The auxiliary directory is a subdirectory of the data directory, and various Cockroach subsystems write to it directly without any incremental accounting of their disk usage. As a result, Capacity calculations walk the auxiliary directory to calculate the directory's disk usage anew each time. Previously, this calculation occurred during every call to Capacity. This commit adapts Pebble.Capacity to cache the computed capacity and recalculate it at most every minute. Ideally, we would incrementally account disk usage within the auxiliary directory (cockroachdb#96344). This change is a stopgap, avoiding the bulk of the CPU usage incurred by Capacity recalculations during IMPORTs. Epic: none Release note: none
jbowens
left a comment
jbowens
left a comment
There was a problem hiding this comment.
TFTR!
Reviewable status:
pkg/storage/pebble.go line 812 at r1 (raw file):
Previously, RaduBerinde wrote…
[nit] crtime.Mono
doesn't time.Time have a monotonic time component that it uses for Sub/Since? I'm forgetting, in what circumstances does crtime.Mono provide an additional benefit?
pkg/storage/pebble.go line 1991 at r1 (raw file):
Previously, RaduBerinde wrote…
[nit] s/crashing/erroring out
Done
RaduBerinde
left a comment
RaduBerinde
left a comment
There was a problem hiding this comment.
Reviewable status:
pkg/storage/pebble.go line 812 at r1 (raw file):
Previously, jbowens (Jackson Owens) wrote…
doesn't time.Time have a monotonic time component that it uses for
Sub/Since? I'm forgetting, in what circumstances doescrtime.Monoprovide an additional benefit?
It's ~2x faster since it only needs to get the monotonic clock, so it's good practice to use. It's also more self-documenting, in that it shows that you only use it to measure elapsed time.
3 participants
Pebble incrementally accounts its disk usage and this accounting is used to cheaply service Capacity calls. The auxiliary directory is a subdirectory of the data directory, and various Cockroach subsystems write to it directly without any incremental accounting of their disk usage. As a result, Capacity calculations walk the auxiliary directory to calculate the directory's disk usage anew each time.
Previously, this calculation occurred during every call to Capacity. This commit adapts Pebble.Capacity to cache the computed capacity and recalculate it at most every minute. Ideally, we would incrementally account disk usage within the auxiliary directory (#96344). This change is a stopgap, avoiding the bulk of the CPU usage incurred by Capacity recalculations during IMPORTs.
Epic: none
Release note: none