Skip to content

Commit 2591f16

Browse files
authored
Merge pull request #77 from inkade/master
Update README.md
2 parents 4961fe7 + f4ce43f commit 2591f16

File tree

1 file changed

+78
-93
lines changed

1 file changed

+78
-93
lines changed

README.md

Lines changed: 78 additions & 93 deletions
Original file line numberDiff line numberDiff line change
@@ -8,35 +8,20 @@
88

99
# Discussion: Compositional Loggers
1010

11-
LoggingExtras is designs around allowing you to build arbitrarily complicated
12-
systems for "log plumbing". That is to say basically routing logged information to different places.
13-
It is built around the idea of simple parts which are composed together,
14-
to allow for powerful and flexible definition of your logging system.
15-
Without you having to define any custom loggers by subtyping `AbstractLogger`.
16-
When we talk about composability we mean to say that the composition of any set of Loggers is itself a Logger.
17-
LoggingExtras is a composable logging system.
18-
19-
Loggers can be broken down into 4 types:
20-
- *Sinks*: Sinks are the final end point of a log messages journey. They write it to file, or display it on the console, or set off a red flashing light in the laboratory. A Sink should never decide what to accept, only what to do with it.
21-
- *Filters*: Filters wrap around other loggers and decide whether or not to pass on a message. They can further be broken down by when that decision occurs (See `ActiveFilteredLogger` vs `EarlyFilteredLogger`).
22-
- *Transformers*: Transformers modify the content of log messages, before passing them on. This includes the metadata like severity level. Unlike Filters they can't block a log message, but they could drop its level down to say `Debug` so that normally noone would see it.
23-
- *Demux*: There is only one possible Demux Logger. and it is central to log routing. It acts as a hub that receives 1 log message, and then sends copies of it to all its child loggers. Like in the diagram above, it can be composed with Filters to control what goes where.
24-
25-
This is a basically full taxonomy of all compositional loggers.
26-
This package implements the full set. So you shouldn't need to build your own routing components, just configure the ones included in this package.
27-
28-
It is worth understanding the idea of logging purity.
29-
The loggers defined in this package are all pure.
30-
The Filters, only filter, the Sinks only sink, the transformers only Transform.
31-
32-
We can contrast this to the the `ConsoleLogger` (the standard logger in the REPL).
33-
The `ConsoleLogger` is an impure sink.
34-
As well as displaying logs to the user (as a Sink);
35-
it uses the log content, in the form of the `max_log` kwarg to decide if a log should be displayed (Active Filtering);
36-
and it has a min_enabled_level setting, that controls if it will accept a message at all
37-
(Early Filtering, in particular see `MinLevelLogger`).
38-
If it was to be defined in a compositional way,
39-
we would write something along the lines of:
11+
LoggingExtras allows routing logged information to different places when constructing complicated "log plumbing" systems. Built upon the concept of simple parts composed together, subtyping `AbstractLogger` provides a powerful and flexible definition for your logging system without a need to define any custom loggers. When we talk about composability, the composition of any set of Loggers is itself a Logger, and LoggingExtras is a composable logging system.
12+
13+
Loggers break down into four types:
14+
- *Sinks*: Sinks are the endpoint of a log message journey. They write it to file or display it on the console or set off a red flashing light in the laboratory. A Sink should never decide what to accept, only what to do with it.
15+
- *Filters*: Filters wrap around other loggers and decide whether or not to pass on a message. When that decision occurs, they can be further broken down (See `ActiveFilteredLogger` vs `EarlyFilteredLogger`).
16+
- *Transformers*: Transformers modify the content of log messages before passing them on, including metadata, such as severity level. Unlike Filters, they can't block a log message, but they could drop its level down to say Debug so that usually no one would see it.
17+
- *Demux*: There is only one possible Demux Logger, and it is central to log routing. It acts as a hub that receives one log message and then sends copies to all its child loggers. Like in the diagram above, it can be composed with Filters to control what goes where.
18+
19+
This is a complete taxonomy of all compositional loggers, with this package implementing the entire set. As such, there should be no need to build routing components when configuring the ones included in this package.
20+
21+
It is worth understanding the idea of logging purity. The loggers defined in this package are all pure. The Filters only filter, the Sinks only sink, and the transformers only Transform.
22+
23+
We can contrast this to the `ConsoleLogger` (the standard logger in the REPL). The `ConsoleLogger` is an impure sink. As well as displaying logs to the user (as a Sink); it uses the log content, in the form of the `max_log`, to decide whether to display a log (Active Filtering); and it has a min_enabled_level setting that controls if it will accept a message at all (Early Filtering, in particular, see `MinLevelLogger`).
24+
If defined in a compositional way, we would write something along the lines of:
4025
```julia
4126
ConsoleLogger(stream, min_level) =
4227
MinLevelLogger(
@@ -49,14 +34,14 @@ ConsoleLogger(stream, min_level) =
4934

5035

5136
# Usage
52-
Load the package with `using LoggingExtras`.
37+
Load the package `using LoggingExtras`.
5338
For convenience, this also re-exports the `Logging` standard library.
5439

5540

5641
### Basics of working with loggers
5742
For full details, see the [Julia documentation on Logging](https://docs.julialang.org/en/v1/stdlib/Logging/index.html)
5843

59-
To use a `logger` in a given scope do
44+
To use a `logger` in a given scope, do
6045
```julia
6146
with_logger(logger) do
6247
#things
@@ -80,45 +65,44 @@ logger = global_logger()
8065

8166

8267
# Loggers introduced by this package:
83-
This package introduces 8 new loggers.
84-
The `TeeLogger`, the `TransformerLogger`, 3 types of filtered logger, the `FileLogger`,
68+
This package introduces eight new loggers;
69+
the `TeeLogger`, the `TransformerLogger`, and three types of filtered logger, the `FileLogger`,
8570
the `DatetimeRotatingFileLogger` and the `FormatLogger`.
8671
All of them, except `FormatLogger`, just wrap existing loggers.
8772
- The `TeeLogger` sends the logs to multiple different loggers.
88-
- The 3 filter loggers are used to control if a message is written or not
89-
- The `MinLevelLogger` only allows messages to pass that are above a given level of severity
90-
- The `EarlyFilteredLogger` lets you write filter rules based on the `level`, `module`, `group` and `id` of the log message
91-
- The `ActiveFilteredLogger` lets you filter based on the full content
73+
- The filter loggers control whether to write a message or not.
74+
- The `MinLevelLogger` only allows messages to pass above a given severity level.
75+
- The `EarlyFilteredLogger` lets you write filter rules based on the log message's `level`, `module`, `group` and `id`.
76+
- The `ActiveFilteredLogger` lets you filter based on the full content.
9277
- The `TransformerLogger` applies a function to modify log messages before passing them on.
9378
- The `FileLogger` is a simple logger sink that writes to file.
9479
- The `DatetimeRotatingFileLogger` is a logger sink that writes to file, rotating logs based upon a user-provided `DateFormat`.
95-
- The `FormatLogger` is a logger sink that simply formats the message and writes to the logger stream.
80+
- The `FormatLogger` is a logger sink that formats the message and writes to the logger stream.
9681
- The `LevelOverrideLogger` for overriding the log level of other loggers
9782

98-
By combining `TeeLogger` with filter loggers you can arbitrarily route log messages, wherever you want.
83+
By combining `TeeLogger` with filter loggers, you can arbitrarily route log messages wherever you want.
9984

10085

10186
## `TeeLogger` (*Demux*)
10287

10388
The `TeeLogger` sends the log messages to multiple places.
10489
It takes a list of loggers.
10590
You often want to pass the `current_logger()` or `global_logger()`
106-
as one of those inputs so it keeps going to that one as well.
91+
as one of those inputs, so it also keeps going to that one.
10792

108-
It is up to those loggers to determine if they will accept it.
109-
Which they do using their methods for `shouldlog` and `min_enabled_level`.
110-
Or you can do, by wrapping them in a filtered logger as discussed below.
93+
It is up to those loggers to determine if they will accept it, which they do by using their methods for `shouldlog` and `min_enabled_level`.
94+
Or you can wrap them in a filtered logger, as discussed below.
11195

11296
## `ActiveFilteredLogger` (*Filter*)
11397

114-
The `ActiveFilteredLogger` exists to give more control over which messages should be logged.
115-
It warps any logger, and before sending messages to the logger to log,
98+
The `ActiveFilteredLogger` exists to give more control over the messages logged.
99+
It warps any logger and, before sending messages to the logger to log,
116100
checks them against a filter function.
117101
The filter function takes the full set of parameters of the message.
118102
(See it's docstring with `?ActiveFilteredLogger` for more details.)
119103

120104
### Demo
121-
We want to filter to only log strings staring with `"Yo Dawg!"`.
105+
We want to filter only log strings starting with `"Yo Dawg!"`.
122106

123107
```julia
124108
julia> function yodawg_filter(log_args)
@@ -141,8 +125,9 @@ end
141125
142126
### Respecting `maxlog` convention
143127
144-
An `ActiveFilterLogger` can be used to wrap another logger to obey `maxlog` directives, for example,
145-
similar to the `make_throttled_logger` example below,
128+
An `ActiveFilterLogger` can be used to wrap another logger to obey `maxlog` directives; for example,
129+
similar to the `make_throttled_logger` example below, it wraps another logger to filter logs that have already fired `maxlog` many times.
130+
See <https://docs.julialang.org/en/v1/stdlib/Logging/#Logging.@logmsg> for more on `maxlog`.
146131
```julia
147132
function make_maxlog_logger(logger)
148133
counts = Dict{Any,Int}()
@@ -160,22 +145,21 @@ function make_maxlog_logger(logger)
160145
end
161146
end
162147
```
163-
wraps another logger to filter logs that have already fired `maxlog` many times.
164-
See <https://docs.julialang.org/en/v1/stdlib/Logging/#Logging.@logmsg> for more on `maxlog`.
148+
165149
166150
## `EarlyFilteredLogger` (*Filter*)
167151
168-
The `EarlyFilteredLogger` is similar to the `ActiveFilteredLogger`,
169-
but it runs earlier in the logging pipeline.
170-
In particular it runs before the message is computed.
171-
It can be useful to filter things early if creating the log message is expensive.
152+
The `EarlyFilteredLogger` is similar to the `ActiveFilteredLogger`
153+
but runs earlier in the logging pipeline.
154+
In particular, it runs before the computation of the message.
155+
It can be useful to filter things early if creating the log message is expensive,
172156
E.g. if it includes summary statistics of the error.
173157
The filter function for early filter logging only has access to the
174158
`level`, `_module`, `id` and `group` fields of the log message.
175-
The most notable use of it is to filter based on modules,
159+
Its most notable use is filtering based on modules;
176160
see the HTTP example below.
177161
178-
Another example is using them to stop messages every being repeated within a given time period.
162+
Another example is using them to stop repeated messages within a given period.
179163
180164
```julia
181165
using Dates, LoggingExtras
@@ -214,8 +198,8 @@ end
214198
```
215199
216200
## `MinLevelLogger` (*Filter*)
217-
This is basically a special case of the early filtered logger,
218-
that just checks if the level of the message is above the level specified when it was created.
201+
This is a special case of the early filtered logger
202+
that checks if the message level is above the level specified when created.
219203
220204
### Demo: filter out all the log messages that are less severe than `Error`
221205
@@ -235,12 +219,12 @@ julia> with_logger(error_only_logger) do
235219
236220
## `TransformerLogger` (*Transformer*)
237221
The transformer logger allows for the modification of log messages.
238-
This modification includes such things as its log level, and content,
222+
This modification includes its log level, content,
239223
and all the other arguments passed to `handle_message`.
240224
241-
When constructing a `TransformerLogger` you pass in a transformation function,
242-
and a logger to be wrapped.
243-
The transformation function takes a named tuple containing all the log message fields,
225+
When constructing a `TransformerLogger` you pass in a transformation function
226+
and a logger to wrap.
227+
The transformation function takes a named tuple containing all the log message fields
244228
and should return a new modified named tuple.
245229
246230
A simple example of its use is truncating messages.
@@ -267,26 +251,26 @@ end
267251
[ Info: Not like this one, that is is short
268252
```
269253
270-
It can also be used to do things such as change the log level of messages from a particular module (see the example below).
254+
`TransformerLogger` can also be used to do things such as change the log level of messages from a particular module (see the example below).
271255
Or to set common properties for all log messages within the `with_logger` block,
272-
for example to set them all to the same `group`.
256+
for example, to set them all to the same `group`.
273257
274258
## `FileLogger` (*Sink*)
275259
The `FileLogger` does logging to file.
276-
It is just a convenience wrapper around the base julia `SimpleLogger`,
277-
to make it easier to pass in a filename, rather than a stream.
278-
It is really simple.
260+
It is just a convenience wrapper around the base Julia `SimpleLogger`,
261+
to make it easier to pass in a filename rather than a stream.
262+
It is straghtforward:
279263
- It takes a filename,
280-
- a kwarg to check if should `always_flush` (default: `true`).
281-
- a kwarg to `append` rather than overwrite (default `false`. i.e. overwrite by default)
282-
The resulting file format is similar to that which is shown in the REPL.
264+
- It uses a kwarg to check if it should `always_flush` (default: `true`).
265+
- Uses a kwarg to `append` rather than overwrite (default `false`. i.e. overwrite by default).
266+
The resulting file format is similar to that shown in the REPL.
283267
(Not identical, but similar)
284268
285-
**NOTE**: To print to the file in a specific format, e.g. to create a JSON log, use
269+
**NOTE**: To print the file in a specific format, e.g. to create a JSON log, use
286270
`FormatLogger` instead.
287271
288272
### Demo: `TeeLogger` and `FileLogger`
289-
We are going to log info and above to one file,
273+
We will log info and above to one file,
290274
and warnings and above to another.
291275
292276
```julia
@@ -322,8 +306,8 @@ shell> cat info.log
322306
323307
## `DatetimeRotatingFileLogger` (*Sink*)
324308
Use this sink to rotate your logs based upon a given `DateFormat`, automatically closing one file and opening another
325-
when the `DateFormat` would change the filename. Note that if you wish to have static portions of your filename, you must
326-
escape them so they are not interpreted by the `DateFormat` code. Example:
309+
when the `DateFormat` changes the filename. Note that if you wish to have static portions of your filename, you must
310+
escape them to prevent interpretation by the `DateFormat` code. Example:
327311
328312
```julia
329313
julia> using LoggingExtras
@@ -342,19 +326,19 @@ julia> filter(f -> endswith(f, ".log"), readdir(pwd()))
342326
"access-2020-07-13-13-25.log"
343327
```
344328
345-
The user implicitly controls when the files will be rolled over based on the `DateFormat` given.
346-
To post-process the newly rotated file pass `rotation_callback::Function` as a keyword argument.
329+
The user implicitly controls when the files are rolled over based on the `DateFormat` given.
330+
To post-process the newly rotated file, pass `rotation_callback::Function` as a keyword argument.
347331
See the docstring with (`?DatetimeRotatingFileLogger` in the REPL) for more details.
348332
349-
To control the logging output it is possible to pass a formatter function as the first argument
350-
in the constructor. See `FormatLogger` for the requirements on the formatter function.
333+
To control the logging output, passing a formatter function as the first argument in the constructor
334+
is possible. See FormatLogger for the requirements on the formatter function.
351335
352336
## `FormatLogger` (*Sink*)
353-
The `FormatLogger` is a sink that formats the message and prints to a wrapped IO.
354-
Formatting is done by providing a function `f(io::IO, log_args::NamedTuple)`.
337+
The `FormatLogger` is a sink that formats the message and prints it to a wrapped IO
338+
with formatting provided by providing a function `f(io::IO, log_args::NamedTuple)`.
355339
356-
`FormatLogger` can take as its second argument either a writeable `IO` or a filepath. The `append::Bool` keyword
357-
argument determines whether the file is opened in append mode (`"a"`) or truncate mode (`"w"`).
340+
`FormatLogger` can take either a writeable `IO` or a filepath as its second argument. The `append::Bool` keyword
341+
argument determines whether to open the file in append mode (`"a"`) or truncate mode (`"w"`).
358342
359343
```julia
360344
julia> using LoggingExtras
@@ -378,9 +362,10 @@ Main | [Warn] This is a warning, should take a look.
378362
## `LevelOverrideLogger` (*Filter*)
379363
Allows overriding the minimum log level set by the logger it wraps.
380364
Useful when debug logging
381-
and used in conjuction with `Logging.with_logger` or `LoggingExtras.withlevel` to
365+
and used in conjunction with `Logging.with_logger` or `LoggingExtras.withlevel` to
382366
temporarily modify the current logger with a custom level.
383-
More generally useful if you want to use the current/global logger as a _sink_ but don't know if it is going to have a problematically high min log level set (as julia's default logger sets min level to `Info`).
367+
More generally applicable if you want to use the current/global logger as a _sink_
368+
but don't know if it will have a problematically high min log level set (as julia's default logger sets min level to `Info`).
384369
385370
```julia
386371
julia> using LoggingExtras
@@ -395,24 +380,24 @@ julia> with_logger(logger) do
395380
```
396381
This is roughly complementary to the `MinLevelFilterLogger`.
397382
The `MinLevelFilterLogger` lets you effectively *raise* the level of any logger it wraps to meet the level you specify.
398-
The `LevelOverrideLogger` lets you *lower* (or *raise*) the level of the wrapped logger as it bypasses checks on it entirely.
383+
The `LevelOverrideLogger` enables you to *lower* (or *raise*) the level of the wrapped logger as it bypasses checks on it entirely.
399384
400385
# Utilities
401386
402387
## Verbosity macros
403-
Sometimes when logging, it is desirable to be able to specify a verbosity level along with
404-
the log level, and to be able to filter on verbosity levels. For example, you may want multiple levels
405-
of verbosity for `Debug` log statements. LoggingExtras.jl exports verbosity macros that act like their
406-
non-verbose counterparts, but allow specifying a verbosity level as well:
388+
Sometimes when logging, it is desirable to specify a verbosity level along with
389+
the log level and to be able to filter on verbosity levels. For example, you may want multiple
390+
verbosity levels for `Debug` log statements. LoggingExtras.jl exports verbosity macros that act like their
391+
non-verbose counterparts but allow specifying a verbosity level as well:
407392
* `@debugv N msg`
408393
* `@infov N msg`
409394
* `@warnv N msg`
410395
* `@errorv N msg`
411396
412-
For verbosity filtering, the `LoggingExtras.withlevel(f, Info; verbosity=0)` utlility is provided
413-
for temporarily (i.e. while `f()` is executed) allowing log messages with `level` and `verbosity`.
414-
This is very handy for allowing finer grained control in debug logging for long-running or complex user API function
415-
calls. For example:
397+
For verbosity filtering, the `LoggingExtras.withlevel(f, Info; verbosity=0)` utility is provided,
398+
temporarily (i.e. while `f()` is executed) allowing log messages with `level` and `verbosity`.
399+
This is very handy for allowing finer-grained debug logging control for long-running or complex user API function calls.
400+
For example:
416401
417402
```julia
418403
using LoggingExtras

0 commit comments

Comments
 (0)