Skip to content

Commit 728afed

Browse files
mdh1418mdh1418
committed
Address Feedback
1 parent 6c9fdb6 commit 728afed

File tree

2 files changed

+148
-89
lines changed

2 files changed

+148
-89
lines changed

‎docs/core/diagnostics/dotnet-trace.md‎

Lines changed: 147 additions & 88 deletions
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,8 @@ The `dotnet-trace` tool:
4343
* Is a cross-platform .NET Core tool.
4444
* Enables the collection of .NET Core traces of a running process without a native profiler.
4545
* Is built on [`EventPipe`](./eventpipe.md) of the .NET Core runtime.
46-
* On Linux, provides additional integration with kernel user_events for native tracing tool compatibility.
46+
* Delivers the same experience on Windows, Linux, and macOS.
47+
* Offers an additional command on Linux to collect Linux perf events.
4748

4849
## Options
4950

@@ -126,7 +127,7 @@ dotnet-trace collect
126127
| ------------ | ------------------- |
127128
| `gc` | `0x1` |
128129
| `gchandle` | `0x2` |
129-
| `fusion` | `0x4` |
130+
| `assemblyloader` | `0x4` |
130131
| `loader` | `0x8` |
131132
| `jit` | `0x10` |
132133
| `ngen` | `0x20` |
@@ -145,7 +146,7 @@ dotnet-trace collect
145146
| `gcheapdump` | `0x100000` |
146147
| `gcsampledobjectallocationhigh` | `0x200000` |
147148
| `gcheapsurvivalandmovement` | `0x400000` |
148-
| `gcheapcollect` | `0x800000` |
149+
| `managedheapcollect` | `0x800000` |
149150
| `gcheapandtypenames` | `0x1000000` |
150151
| `gcsampledobjectallocationlow` | `0x2000000` |
151152
| `perftrack` | `0x20000000` |
@@ -159,7 +160,10 @@ dotnet-trace collect
159160
| `compilationdiagnostic` | `0x2000000000` |
160161
| `methoddiagnostic` | `0x4000000000` |
161162
| `typediagnostic` | `0x8000000000` |
163+
| `jitinstrumentationdata` | `0x10000000000` |
164+
| `profiler` | `0x20000000000` |
162165
| `waithandle` | `0x40000000000` |
166+
| `allocationsampling` | `0x80000000000` |
163167

164168
You can read about the CLR provider more in detail on the [.NET runtime provider reference documentation](../../fundamentals/diagnostics/runtime-events.md).
165169

@@ -213,7 +217,9 @@ dotnet-trace collect
213217

214218
| Profile | Description |
215219
|---------|-------------|
216-
|`cpu-sampling`|Useful for tracking CPU usage and general .NET runtime information. This is the default option if no profile or providers are specified.|
220+
|`dotnet-default`|Combination of `dotnet-thread-time` and `dotnet-clr-default`. This is the default profile used when no other provider/event/profile options are specified.|
221+
|`dotnet-thread-time`|Useful for tracking .NET thread-time information. This is the default profile used when no other provider/event/profile options are specified.|
222+
|`dotnet-clr-default`|Useful for tracking a multitude of .NET events.|
217223
|`gc-verbose`|Tracks GC collections and samples object allocations.|
218224
|`gc-collect`|Tracks GC collections only at very low overhead.|
219225

@@ -268,131 +274,184 @@ dotnet-trace collect
268274
269275
## dotnet-trace collect-linux
270276

271-
Collects diagnostic traces from .NET applications using Linux user_events as a transport layer. This command provides the same functionality as [`dotnet-trace collect`](#dotnet-trace-collect) but routes .NET runtime events through the Linux kernel's user_events subsystem before writing them to `.nettrace` files.
277+
Collects diagnostic traces using perf_events, a Linux OS technology. `collect-linux` requires admin privileges to capture kernel- and user-mode events, and by default, captures events from all processes.
272278

273-
This transport approach enables automatic unification of user-space .NET events with kernel-space system events, since both are captured in the same kernel tracing infrastructure. Linux tools like `perf` and `ftrace` can monitor events in real-time while maintaining full compatibility with existing .NET profiling workflows.
279+
This Linux-only command includes the same .NET events as [`dotnet-trace collect`](#dotnet-trace-collect), and it uses the kernel’s user_events mechanism to emit .NET events as perf events, enabling unification of user-space .NET events with kernel-space system events.
274280

275281
### Prerequisites
276282

277283
- Linux kernel with `CONFIG_USER_EVENTS=y` support (kernel 6.4+)
278-
- Appropriate permissions to access `/sys/kernel/tracing/user_events_data`
284+
- Root permissions
279285
- .NET 10+
280286

281287
### Synopsis
282288

283289
```dotnetcli
284290
dotnet-trace collect-linux
285-
[--buffersize <size>]
291+
[-h|--help]
292+
293+
# Provider/Event Specification
294+
[--providers <list-of-comma-separated-providers>]
286295
[--clreventlevel <clreventlevel>]
287296
[--clrevents <clrevents>]
297+
[--perf-event-tracepoints <list-of-perf-event-tracepoints>]
298+
[--profile <profile-name>]
299+
300+
# Trace Collection
288301
[--format <Chromium|NetTrace|Speedscope>]
289-
[-h|--help]
302+
[-o|--output <trace-file-path>]
290303
[--duration dd:hh:mm:ss]
304+
305+
# .NET Process Target (Optional)
291306
[-n, --name <name>]
292-
[--diagnostic-port]
293-
[-o|--output <trace-file-path>]
294307
[-p|--process-id <pid>]
295-
[--profile <profile-name>]
296-
[--providers <list-of-comma-separated-providers>]
297-
[-- <command>] (for target applications running .NET 10 or later)
308+
[-- <command>]
298309
[--show-child-io]
299-
[--resume-runtime]
300-
[--stopping-event-provider-name <stoppingEventProviderName>]
301-
[--stopping-event-event-name <stoppingEventEventName>]
302-
[--stopping-event-payload-filter <stoppingEventPayloadFilter>]
303-
[--tracepoint-configs <list-of-comma-separated-tracepoint-configs>]
304-
[--kernel-events <list-of-kernel-events>]
305310
```
306311

307312
### Options
308313

309-
`dotnet-trace collect-linux` supports all the same options as [`dotnet-trace collect`](#dotnet-trace-collect), excluding `--dsrouter`, and additionally offers:
314+
#### Provider/Event Specification Options
310315

311-
- **`--tracepoint-configs <list-of-comma-separated-tracepoint-configs>` (required)**
316+
- **`--providers <list-of-comma-separated-providers>`**
312317

313-
Defines the explicit mapping between EventPipe providers and kernel tracepoints. Each provider in `--providers` must have a corresponding entry in `--tracepoint-configs`
318+
A comma-separated list of `EventPipe` providers to be enabled. These providers supplement any providers implied by `--profile <profile-name>`. If there's any inconsistency for a particular provider, this configuration takes precedence over the implicit configuration from the profile.
314319

315-
**Format:** `ProviderName:<DefaultTracepointName>:<TracepointSets>`
320+
This list of providers is in the form:
316321

317-
Where:
318-
- `ProviderName`: The EventPipe provider name (e.g., `Microsoft-Windows-DotNETRuntime`)
319-
- `DefaultTracepointName`: Default tracepoint name for this provider (can be empty to require explicit assignment)
320-
- `TracepointSets`: Semi-colon delimited `TracepointName=<EventIds>`
321-
- `EventIds`: Plus-delimited event IDs to route to that tracepoint
322+
- `Provider[,Provider]`
323+
- `Provider` is in the form: `KnownProviderName[:Flags[:Level][:KeyValueArgs]]`.
324+
- `KeyValueArgs` is in the form: `[key1=value1][;key2=value2]`.
322325

323-
> [!NOTE]
324-
> All tracepoint names are automatically prefixed with the provider name to avoid collisions. For example, `gc_events` for the `Microsoft-Windows-DotNETRuntime` provider becomes `Microsoft_Windows_DotNETRuntime_gc_events`.
326+
To learn more about some of the well-known providers in .NET, refer to [Well-known Event Providers](./well-known-event-providers.md).
325327

326-
**Examples:**
327-
```
328-
# Scenario: All events from provider go to a default tracepoint
329-
--tracepoint-configs "Microsoft-Windows-DotNETRuntime:dotnet_runtime"
330-
# All enabled events from Microsoft-Windows-DotNETRuntime will be written to Microsoft_Windows_DotNETRuntime_dotnet_runtime
331-
332-
# Scenario: Split events by categories
333-
--tracepoint-configs "Microsoft-Windows-DotNETRuntime::gc_events=1+2+3;jit_events=10+11+12"
334-
# EventIDs 1, 2, and 3 will be written to Microsoft_Windows_DotNETRuntime_gc_events
335-
# EventIDs 10, 11, and 12 will be written to Microsoft_Windows_DotNETRuntime_jit_events
336-
337-
# Multiple providers (comma-separated)
338-
--tracepoint-configs "Microsoft-Windows-DotNETRuntime::gc_events=1+2+3,MyCustomProvider:custom_events"
339-
# EventIds 1, 2, and 3 from Microsoft-Windows-DotNETRuntime will be written to Microsoft_Windows_DotNETRuntime_gc_events
340-
# All enabled events from MyCustomProvider will be written to MyCustomProvider_custom_events
341-
```
328+
- **`--clreventlevel <clreventlevel>`**
342329

343-
- **`--kernel-events <list-of-kernel-events>` (optional)**
330+
Verbosity of CLR events to be emitted.
331+
The following table shows the available event levels.
344332

345-
A comma-separated list of kernel event categories to include in the trace. These events are automatically grouped into kernel-named tracepoints. Available categories include:
333+
| String value | Numeric value |
334+
| --------------- | :-----------: |
335+
| `logalways` | `0` |
336+
| `critical` | `1` |
337+
| `error` | `2` |
338+
| `warning` | `3` |
339+
| `informational` | `4` |
340+
| `verbose` | `5` |
346341

347-
| Category | Description | Linux Tracepoints |
348-
|----------|-------------|-------------------|
349-
| `syscalls` | System call entry/exit events | `syscalls:sys_enter_*`, `syscalls:sys_exit_*` |
350-
| `sched` | Process scheduling events | `sched:sched_switch`, `sched:sched_wakeup` |
351-
| `net` | Network-related events | `net:netif_rx`, `net:net_dev_xmit` |
352-
| `fs` | Filesystem I/O events | `ext4:*`, `vfs:*` |
353-
| `mm` | Memory management events | `kmem:*`, `vmscan:*` |
342+
- **`--clrevents <clrevents>`**
354343

355-
These events correspond to Linux kernel tracepoints documented in the [Linux kernel tracing documentation](https://www.kernel.org/doc/html/latest/trace/index.html). For more details on available tracepoints, see [ftrace](https://www.kernel.org/doc/html/latest/trace/ftrace.html) and [tracepoints](https://www.kernel.org/doc/html/latest/trace/tracepoints.html).
344+
A list of CLR runtime provider keywords to enable separated by `+` signs. This is a simple mapping that lets you specify event keywords via string aliases rather than their hex values. For example, `dotnet-trace collect --providers Microsoft-Windows-DotNETRuntime:3:4` requests the same set of events as `dotnet-trace collect --clrevents gc+gchandle --clreventlevel informational`. The table below shows the list of available keywords:
356345

357-
Example: `--kernel-events syscalls,sched,net`
346+
| Keyword String Alias | Keyword Hex Value |
347+
| ------------ | ------------------- |
348+
| `gc` | `0x1` |
349+
| `gchandle` | `0x2` |
350+
| `assemblyloader` | `0x4` |
351+
| `loader` | `0x8` |
352+
| `jit` | `0x10` |
353+
| `ngen` | `0x20` |
354+
| `startenumeration` | `0x40` |
355+
| `endenumeration` | `0x80` |
356+
| `security` | `0x400` |
357+
| `appdomainresourcemanagement` | `0x800` |
358+
| `jittracing` | `0x1000` |
359+
| `interop` | `0x2000` |
360+
| `contention` | `0x4000` |
361+
| `exception` | `0x8000` |
362+
| `threading` | `0x10000` |
363+
| `jittedmethodiltonativemap` | `0x20000` |
364+
| `overrideandsuppressngenevents` | `0x40000` |
365+
| `type` | `0x80000` |
366+
| `gcheapdump` | `0x100000` |
367+
| `gcsampledobjectallocationhigh` | `0x200000` |
368+
| `gcheapsurvivalandmovement` | `0x400000` |
369+
| `managedheapcollect` | `0x800000` |
370+
| `gcheapandtypenames` | `0x1000000` |
371+
| `gcsampledobjectallocationlow` | `0x2000000` |
372+
| `perftrack` | `0x20000000` |
373+
| `stack` | `0x40000000` |
374+
| `threadtransfer` | `0x80000000` |
375+
| `debugger` | `0x100000000` |
376+
| `monitoring` | `0x200000000` |
377+
| `codesymbols` | `0x400000000` |
378+
| `eventsource` | `0x800000000` |
379+
| `compilation` | `0x1000000000` |
380+
| `compilationdiagnostic` | `0x2000000000` |
381+
| `methoddiagnostic` | `0x4000000000` |
382+
| `typediagnostic` | `0x8000000000` |
383+
| `jitinstrumentationdata` | `0x10000000000` |
384+
| `profiler` | `0x20000000000` |
385+
| `waithandle` | `0x40000000000` |
386+
| `allocationsampling` | `0x80000000000` |
358387

359-
### Linux Integration
388+
You can read about the CLR provider more in detail on the [.NET runtime provider reference documentation](../../fundamentals/diagnostics/runtime-events.md).
360389

361-
**Tracepoint Configuration Requirements:**
390+
- **`--perf-event-tracepoints <list-of-perf-event-tracepoints>`**
362391

363-
- **Mandatory Mapping**: Every provider must be explicitly mapped to at least a default tracepoint and/or exclusive tracepoint sets via `--tracepoint-configs`
364-
- **Tracepoint Isolation**: Each tracepoint can only receive events from one provider
365-
- **Event Routing**: Different event IDs within a provider can be routed to different tracepoints for granular control
366-
- **Automatic Prefixing**: All tracepoint names are prefixed with the provider name to avoid collisions
392+
A comma-separated list of kernel perf event tracepoints to include in the trace. Available kernel events can be found under tracefs, which is typically mounted at `/sys/kernel/tracing`, through `available_events` for all available tracepoints or through the `events/` subdirectory for categorized events.
367393

368-
**Kernel Integration Points:**
394+
Example: `--perf-event-tracepoints syscalls:sys_enter_execve,sched:sched_switch,sched:sched_wakeup`
369395

370-
The kernel tracepoints can be accessed through standard Linux tracing interfaces:
396+
- **`--profile <profile-name>`**
371397

372-
- **ftrace**: `/sys/kernel/tracing/events/user_events/`
373-
- **perf**: Use `perf list user_events*` to see available events
374-
- **System monitoring tools**: Any tool that can consume Linux tracepoints
398+
A named pre-defined set of provider configurations that allows common tracing scenarios to be specified succinctly. The following profiles are available:
375399

376-
### Examples
400+
| Profile | Description |
401+
|---------|-------------|
402+
|`dotnet-default`|Combination of `dotnet-thread-time` and `dotnet-clr-default`. This is the default profile used when no other provider/event/profile options are specified.|
403+
|`dotnet-thread-time`|Useful for tracking .NET thread-time information. This is the default profile used when no other provider/event/profile options are specified.|
404+
|`dotnet-clr-default`|Useful for tracking a multitude of .NET events.|
405+
|`gc-verbose`|Tracks GC collections and samples object allocations.|
406+
|`gc-collect`|Tracks GC collections only at very low overhead.|
407+
|`cpu-sampling`|Useful for tracking CPU usage.|
377408

378-
```dotnetcli
379-
# All runtime events to one tracepoint
380-
dotnet-trace collect-linux --process-id 1234 \
381-
--providers Microsoft-Windows-DotNETRuntime:0x8000:5 \
382-
--kernel-events syscalls,sched \
383-
--tracepoint-configs "Microsoft-Windows-DotNETRuntime:dotnet_runtime"
384-
385-
# Split runtime events by category
386-
dotnet-trace collect-linux --process-id 1234 \
387-
--providers Microsoft-Windows-DotNETRuntime:0x8001:5 \
388-
--kernel-events syscalls,sched,net,fs \
389-
--tracepoint-configs "Microsoft-Windows-DotNETRuntime::exception_events=80;gc_events=1+2"
390-
391-
# Multiple providers
392-
dotnet-trace collect-linux --process-id 1234 \
393-
--providers "Microsoft-Windows-DotNETRuntime:0x8001:5,MyCustomProvider:0xFFFFFFFF:5" \
394-
--tracepoint-configs "Microsoft-Windows-DotNETRuntime:dotnet_runtime,MyCustomProvider:custom_events"
395-
```
409+
#### Trace Collection Options
410+
411+
- **`--format {Chromium|NetTrace|Speedscope}`**
412+
413+
Sets the output format for the trace file conversion. The default is `NetTrace`.
414+
415+
- **`-o|--output <trace-file-path>`**
416+
417+
The output path for the collected trace data. If not specified it defaults to `<appname>_<yyyyMMdd>_<HHmmss>.nettrace`, for example, `myapp_20210315_111514.nettrace``.
418+
419+
- **`--duration <time-to-run>`**
420+
421+
The time for the trace to run. Use the `dd:hh:mm:ss` format. For example `00:00:00:05` will run it for 5 seconds.
422+
423+
#### .NET Process Target Options
424+
425+
> ![NOTE]
426+
> The default behavior for collect-linux is to collect .NET events and perf event tracepoints from ALL discoverable processes.
427+
428+
- **`-n, --name <name>`**
429+
430+
The name of the process to collect the trace from.
431+
432+
> [!NOTE]
433+
> By default, `/tmp` will be searched to discover .NET processes diagnostics ports. Should the target application have a `TMPDIR` enviornment variable set, `dotnet-trace` will need to share the same `TMPDIR` value, otherwise events will not be collected from the target application.
434+
435+
- **`-p|--process-id <PID>`**
436+
437+
The process ID to collect the trace from.
438+
439+
> [!NOTE]
440+
> By default, `/tmp` will be searched to discover .NET processes diagnostics ports. Should the target application have a `TMPDIR` enviornment variable set, `dotnet-trace` will need to share the same `TMPDIR` value, otherwise events will not be collected from the target application.
441+
442+
- **`-- <command>`**
443+
444+
After the collection configuration parameters, the user can append `--` followed by a command to start a .NET application. This may be helpful when diagnosing issues that happen early in the process, such as startup performance issue or assembly loader and binder errors.
445+
446+
- **`--show-child-io`**
447+
448+
Only applicable when `-- <command>` is used. Shows the input and output streams of a launched child process in the current console.
449+
450+
> [!NOTE]
451+
452+
> - Stopping the trace may take a long time (up to minutes) for large applications. The runtime needs to send over the type cache for all managed code that was captured in the trace.
453+
454+
> - To collect a trace using `dotnet-trace collect-linux`, it needs to be run with root permissions (`CAP_PERFMON`/`CAP_SYS_ADMIN`). Otherwise, the tool will fail to establish collect events.
396455
397456
## dotnet-trace convert
398457

‎docs/core/diagnostics/eventsource-collect-and-view-traces.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -214,7 +214,7 @@ into other formats, such as Chromium or [Speedscope](https://www.speedscope.app/
214214
Trace completed.
215215
```
216216

217-
dotnet-trace uses a comma-delimited variant of the [conventional text format](#conventions-for-describing-provider-configuration) for describing provider configuration in
217+
dotnet-trace uses the [conventional text format](#conventions-for-describing-provider-configuration) for describing provider configuration in
218218
the `--providers` argument. For more options on how to take traces using dotnet-trace, see the
219219
[dotnet-trace docs](./dotnet-trace.md#collect-a-trace-with-dotnet-trace).
220220

0 commit comments

Comments
 (0)