Skip to content

Commit 16b1ef9

Browse files
docs: Simplify logging basics (#614)
* Refactor logging basics documentation Moved advanced logging details from logging.md to logging-advanced.md. Updated logging.md to focus on basic logging functions (enable_logging!, disable_logging!, fetch_logs!). Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
1 parent 768eb16 commit 16b1ef9

File tree

2 files changed

+62
-35
lines changed

2 files changed

+62
-35
lines changed

docs/src/logging-advanced.md

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,33 @@
11
# Logging: Advanced
22

3+
Dagger's scheduler keeps track of the important and potentially expensive
4+
actions it does, such as moving data between workers or executing thunks, and
5+
tracks how much time and memory allocations these operations consume, among
6+
other things. It does it through the `TimespanLogging.jl` package (which used
7+
to be directly integrated into Dagger). Saving this information somewhere
8+
accessible is disabled by default, but it's quite easy to turn it on, through
9+
two mechanisms.
10+
11+
The first is `Dagger.enable_logging!`, which provides an easy-to-use interface
12+
to both enable and configure logging. The defaults are usually sufficient for
13+
most users, but can be tweaked with keyword arguments.
14+
15+
The second is done by setting a "log sink" in the Dagger `Context` being used,
16+
as `ctx.log_sink`. These log sinks drive how Dagger's logging behaves, and are
17+
configurable by the user, without the need to tweak any of Dagger's internal
18+
code.
19+
20+
A variety of log sinks are built-in to TimespanLogging; the `NoOpLog` is the
21+
default log sink when one isn't explicitly specified, and disables logging
22+
entirely (to minimize overhead). There are currently two other log sinks of
23+
interest; the first and newer of the two is the `MultiEventLog`, which
24+
generates multiple independent log streams, one per "consumer" (details in the
25+
next section). This is the log sink that `enable_logging!` uses, as it's easily
26+
the most flexible. The second and older sink is the `LocalEventLog`, which is
27+
explained later in this document. Most users are recommended to use the
28+
`MultiEventLog` (ideally via `enable_logging!`) since it's far more flexible
29+
and extensible, and is more performant in general.
30+
331
## MultiEventLog
432

533
The `MultiEventLog` is intended to be configurable to exclude unnecessary

docs/src/logging.md

Lines changed: 34 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -1,35 +1,34 @@
1-
# Logging and Graphing
2-
3-
Dagger's scheduler keeps track of the important and potentially expensive
4-
actions it does, such as moving data between workers or executing thunks, and
5-
tracks how much time and memory allocations these operations consume, among
6-
other things. It does it through the `TimespanLogging.jl` package (which used
7-
to be directly integrated into Dagger). Saving this information somewhere
8-
accessible is disabled by default, but it's quite easy to turn it on, through
9-
two mechanisms.
10-
11-
The first is `Dagger.enable_logging!`, which provides an easy-to-use interface
12-
to both enable and configure logging. The defaults are usually sufficient for
13-
most users, but can be tweaked with keyword arguments.
14-
15-
The second is done by setting a "log sink" in the Dagger `Context` being used,
16-
as `ctx.log_sink`. These log sinks drive how Dagger's logging behaves, and are
17-
configurable by the user, without the need to tweak any of Dagger's internal
18-
code.
19-
20-
A variety of log sinks are built-in to TimespanLogging; the `NoOpLog` is the
21-
default log sink when one isn't explicitly specified, and disables logging
22-
entirely (to minimize overhead). There are currently two other log sinks of
23-
interest; the first and newer of the two is the `MultiEventLog`, which
24-
generates multiple independent log streams, one per "consumer" (details in the
25-
next section). This is the log sink that `enable_logging!` uses, as it's easily
26-
the most flexible. The second and older sink is the `LocalEventLog`, which is
27-
explained later in this document. Most users are recommended to use the
28-
`MultiEventLog` (ideally via `enable_logging!`) since it's far more flexible
29-
and extensible, and is more performant in general.
30-
31-
Log sinks are explained in detail in [Logging: Advanced](@ref); however, if
32-
using `enable_logging!`, everything is already configured for you. Then, all
33-
you need to do is call `Dagger.fetch_logs!()` to get the logs for all workers
34-
as a `Dict`. A variety of tools can operate on these logs, including
35-
visualization through `show_logs` and `render_logs`.
1+
# Logging
2+
3+
Dagger provides mechanisms to log and visualize scheduler events. This can be useful for debugging and performance analysis.
4+
5+
## Basic Logging Functions
6+
7+
The primary functions for controlling logging are:
8+
9+
- `Dagger.enable_logging!`: Enables logging. This function uses the `MultiEventLog` by default, which is flexible and performant. You can customize its behavior with keyword arguments.
10+
- `Dagger.disable_logging!`: Disables logging.
11+
- `Dagger.fetch_logs!`: Fetches the logs from all workers. This returns a `Dict` where keys are worker IDs and values are the logs.
12+
13+
## Example Usage
14+
15+
```julia
16+
using Dagger
17+
18+
# Enable logging
19+
Dagger.enable_logging!()
20+
21+
# Run some Dagger computations
22+
wait(Dagger.@spawn sum([1, 2, 3]))
23+
24+
# Fetch logs
25+
logs = Dagger.fetch_logs!()
26+
27+
# Disable logging
28+
Dagger.disable_logging!()
29+
30+
# You can now inspect the `logs` Dict or use visualization tools
31+
# like `show_logs` and `render_logs` (see [Logging: Visualization](@ref logging-visualization.md)).
32+
```
33+
34+
For more advanced logging configurations, such as custom log sinks and consumers, see [Logging: Advanced](@ref).

0 commit comments

Comments
 (0)