Skip to content

Commit 3c7a2cf

Browse files
committed
bug engine: don't hang on 4000+ tasks with stack usage monitor
A bug was found where creating 4000+ tasks at the same time with stack usage monitor enabled would hang the process in release builds. As a bandaid, only enable stack usage monitor for no more than 1000 tasks. This means that some stack overflows in high-load services may have poor diagnostics. commit_hash:f2fbcea2813caffe437adacd9edd6b7271dc64d8
1 parent a1f7525 commit 3c7a2cf

File tree

2 files changed

+7
-1
lines changed

2 files changed

+7
-1
lines changed

core/src/engine/coro/pool.cpp

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -127,7 +127,9 @@ Pool::Coroutine Pool::CreateCoroutine(bool quiet) {
127127
LOG_DEBUG() << "Created a coroutine #" << new_total << '/' << config_.max_size;
128128
}
129129

130-
stack_usage_monitor_.Register(coroutine);
130+
if (new_total <= kStackUsageMonitorLimit) {
131+
stack_usage_monitor_.Register(coroutine);
132+
}
131133

132134
return coroutine;
133135
} catch (const std::bad_alloc&) {

core/src/engine/coro/stack_usage_monitor.hpp

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,10 @@ class StackUsageMonitor final {
4040
utils::FastPimpl<Impl, 1024, 8> impl_;
4141
};
4242

43+
// It was discovered experimentally that registering more than ~4000 coroutines at the same time may crash or hang
44+
// the service on Linux x86_64 after UFFDIO_REGISTER failure.
45+
constexpr std::size_t kStackUsageMonitorLimit = 1000;
46+
4347
std::size_t GetCurrentTaskStackUsageBytes() noexcept;
4448

4549
} // namespace engine::coro

0 commit comments

Comments
 (0)