Adding any kind of profiling other than CPU causes significant performance degredation

[sthomson-wyn]

  CORECLR_ENABLE_PROFILING: "1"
  CORECLR_PROFILER: '{BD1A650D-AC5D-4896-B64F-D6FA25D6B26A}'
  CORECLR_PROFILER_PATH: /Pyroscope.Profiler.Native.so
  LD_PRELOAD: /Pyroscope.Linux.ApiWrapper.x64.so
  PYROSCOPE_PROFILING_ALLOCATION_ENABLED: "false"
  PYROSCOPE_PROFILING_CPU_ENABLED: "true"
  PYROSCOPE_PROFILING_ENABLED: "true"
  PYROSCOPE_PROFILING_EXCEPTION_ENABLED: "false"
  PYROSCOPE_PROFILING_HEAP_ENABLED: "false"
  PYROSCOPE_PROFILING_LOCK_ENABLED: "false"
  PYROSCOPE_PROFILING_WALLTIME_ENABLED: "false"

With the above settings, performance is comparable to an equivalent workload running without any profiling.

  CORECLR_ENABLE_PROFILING: "1"
  CORECLR_PROFILER: '{BD1A650D-AC5D-4896-B64F-D6FA25D6B26A}'
  CORECLR_PROFILER_PATH: /Pyroscope.Profiler.Native.so
  LD_PRELOAD: /Pyroscope.Linux.ApiWrapper.x64.so
  PYROSCOPE_PROFILING_ALLOCATION_ENABLED: "true"
  PYROSCOPE_PROFILING_CPU_ENABLED: "true"
  PYROSCOPE_PROFILING_ENABLED: "true"
  PYROSCOPE_PROFILING_EXCEPTION_ENABLED: "true"
  PYROSCOPE_PROFILING_HEAP_ENABLED: "true"
  PYROSCOPE_PROFILING_LOCK_ENABLED: "true"
  PYROSCOPE_PROFILING_WALLTIME_ENABLED: "true"

With the above settings, performance tanks hard.

Are there any dials we can turn/investigate to mitigate this? Are certain workloads more sensitive than other workloads? I know some overhead is expected, but this renders any profiling other than CPU pretty much unusable

It's basically any combination of profilers. For example: Allocation AND CPU profilers causes performance to tank, but either one on their own is fine