8
8
9
9
# Discussion: Compositional Loggers
10
10
11
- LoggingExtras is designs around allowing you to build arbitrarily complicated
11
+ LoggingExtras is designs around allowing you to build arbitrarily complicated
12
12
systems for "log plumbing". That is to say basically routing logged information to different places.
13
13
It is built around the idea of simple parts which are composed together,
14
14
to allow for powerful and flexible definition of your logging system.
@@ -30,16 +30,15 @@ The loggers defined in this package are all pure.
30
30
The Filters, only filter, the Sinks only sink, the transformers only Transform.
31
31
32
32
We can contrast this to the the ` ConsoleLogger ` (the standard logger in the REPL).
33
- The ` ConsoleLogger ` is an in-pure sink.
33
+ The ` ConsoleLogger ` is an impure sink.
34
34
As well as displaying logs to the user (as a Sink);
35
35
it uses the log content, in the form of the ` max_log ` kwarg to decide if a log should be displayed (Active Filtering);
36
- and it has a min_enabled_level setting, that controls if it will accept a message at all
36
+ and it has a min_enabled_level setting, that controls if it will accept a message at all
37
37
(Early Filtering, in particular see ` MinLevelLogger ` ).
38
38
If it was to be defined in a compositional way,
39
- we would write;
39
+ we would write something along the lines of:
40
40
```
41
-
42
- ConsoleLogger(stream, min_level) =
41
+ ConsoleLogger(stream, min_level) =
43
42
MinLevelLogger(
44
43
ActiveFilteredLogger(max_log_filter,
45
44
PureConsoleLogger(stream)
@@ -76,27 +75,21 @@ logger = global_logger()
76
75
```
77
76
78
77
# Loggers introduced by this package:
79
-
80
-
81
- This package introduces 5 new loggers.
82
- The ` DemuxLogger ` , the ` FileLogger ` , and 3 types of filtered logger.
78
+ This package introduces 6 new loggers.
79
+ The ` DemuxLogger ` , the ` TransformerLogger ` , 3 types of filtered logger, and the ` FileLogger ` .
83
80
All of them just wrap existing loggers.
84
81
- The ` DemuxLogger ` sends the logs to multiple different loggers.
85
82
- The ` TransformerLogger ` applies a function to modify log messages before passing them on.
86
- - The ` FileLogger ` is a simple logger sink that writes to file.
87
83
- The 3 filter loggers are used to control if a message is written or not
88
84
- The ` MinLevelLogger ` only allowes messages to pass that are above a given level of severity
89
85
- The ` EarlyFilteredLogger ` lets you write filter rules based on the ` level ` , ` module ` , ` group ` and ` id ` of the log message
90
86
- The ` ActiveFilteredLogger ` lets you filter based on the full content
91
-
87
+ - The ` FileLogger ` is a simple logger sink that writes to file.
92
88
93
89
By combining ` DemuxLogger ` with filter loggers you can arbitrarily route log messages, wherever you want.
94
90
95
- The ` FileLogger ` is just a convience wrapper around the base julia ` SimpleLogger ` ,
96
- to make it easier to pass in a filename, rather than a stream.
97
-
98
91
99
- ## ` DemuxLogger ` and ` FileLogger `
92
+ ## ` DemuxLogger `
100
93
101
94
The ` DemuxLogger ` sends the log messages to multiple places.
102
95
It takes a list of loggers.
@@ -107,13 +100,18 @@ It is up to those loggers to determine if they will accept it.
107
100
Which they do using their methods for ` shouldlog ` and ` min_enabled_level ` .
108
101
Or you can do, by wrapping them in a filtered logger as discussed below.
109
102
103
+ ## ` FileLogger `
110
104
The ` FileLogger ` does logging to file.
105
+ It is just a convience wrapper around the base julia ` SimpleLogger ` ,
106
+ to make it easier to pass in a filename, rather than a stream.
111
107
It is really simple.
112
- It takes a filename,
108
+ - It takes a filename,
113
109
- a kwarg to check if should ` always_flush ` (default: ` true ` ).
114
110
- a kwarg to ` append ` rather than overwrite (default ` false ` . i.e. overwrite by default)
111
+ The resulting file format is similar to that which is shown in the REPL.
112
+ (Not identical, but similar)
115
113
116
- ### Demo
114
+ ### Demo: ` DemuxLogger ` and ` FileLogger `
117
115
We are going to log info and above to one file,
118
116
and warnings and above to another.
119
117
@@ -123,7 +121,7 @@ julia> using Logging; using LoggingExtras;
123
121
julia> demux_logger = DemuxLogger(
124
122
MinLevelLogger(FileLogger("info.log"), Logging.Info),
125
123
MinLevelLogger(FileLogger("warn.log"), Logging.Warn),
126
- include_current_global=false
124
+ include_current_global=false
127
125
);
128
126
129
127
@@ -164,7 +162,7 @@ We want to filter to only log strings staring with `"Yo Dawg!"`.
164
162
julia> function yodawg_filter(log_args)
165
163
startswith(log_args.message, "Yo Dawg!")
166
164
end
167
- yodawg_filter (generic function with 1 method)
165
+ yodawg_filter (generic function with 1 method)
168
166
169
167
julia> filtered_logger = ActiveFilteredLogger(yodawg_filter, global_logger());
170
168
@@ -186,7 +184,7 @@ but it runs earlier in the logging pipeline.
186
184
In particular it runs before the message is computed.
187
185
It can be useful to filter things early if creating the log message is expensive.
188
186
E.g. if it includes summary statistics of the error.
189
- The filter function for early filter logging only has access to the
187
+ The filter function for early filter logging only has access to the
190
188
` level ` , ` _module ` , ` id ` and ` group ` fields of the log message.
191
189
The most notable use of it is to filter based on modules,
192
190
see the HTTP example below.
@@ -197,35 +195,35 @@ Another example is using them to stop messages every being repeated within a giv
197
195
using Dates, Logging, LoggingExtras
198
196
199
197
julia> function make_throttled_logger(period)
200
- history = Dict{Symbol, DateTime}()
201
- # We are going to use a closure
202
- EarlyFilteredLogger(global_logger()) do log
203
- if !haskey(history, log.id) || (period < now() - history[log.id])
204
- # then we will log it, and update record of when we did
205
- history[log.id] = now()
206
- return true
207
- else
208
- return false
209
- end
210
- end
211
- end
198
+ history = Dict{Symbol, DateTime}()
199
+ # We are going to use a closure
200
+ EarlyFilteredLogger(global_logger()) do log
201
+ if !haskey(history, log.id) || (period < now() - history[log.id])
202
+ # then we will log it, and update record of when we did
203
+ history[log.id] = now()
204
+ return true
205
+ else
206
+ return false
207
+ end
208
+ end
209
+ end
212
210
make_throttled_logger (generic function with 1 method)
213
211
214
212
julia> throttled_logger = make_throttled_logger(Second(3));
215
213
216
214
julia> with_logger(throttled_logger) do
217
- for ii in 1:10
218
- sleep(1)
219
- @info "It happen " ii
220
- end
221
- end
222
- ┌ Info: It happen
215
+ for ii in 1:10
216
+ sleep(1)
217
+ @info "It happened " ii
218
+ end
219
+ end
220
+ ┌ Info: It happened
223
221
└ ii = 1
224
- ┌ Info: It happen
222
+ ┌ Info: It happened
225
223
└ ii = 4
226
- ┌ Info: It happen
224
+ ┌ Info: It happened
227
225
└ ii = 7
228
- ┌ Info: It happen
226
+ ┌ Info: It happened
229
227
└ ii = 10
230
228
```
231
229
@@ -249,19 +247,19 @@ A simple example of its use is truncating messages.
249
247
julia> using Logging, LoggingExtras
250
248
251
249
julia> truncating_logger = TransformerLogger(global_logger()) do log
252
- if length(log.message) > 128
253
- short_message = log.message[1:min(end, 125)] * "..."
254
- return merge(log, (;message=short_message))
255
- else
256
- return log
257
- end
258
- end;
250
+ if length(log.message) > 128
251
+ short_message = log.message[1:min(end, 125)] * "..."
252
+ return merge(log, (;message=short_message))
253
+ else
254
+ return log
255
+ end
256
+ end;
259
257
260
258
julia> with_logger(truncating_logger) do
261
- @info "the truncating logger only truncates long messages"
262
- @info "Like this one that is this is a long and rambling message, it just keeps going and going and going, and it seems like it will never end."
263
- @info "Not like this one, that is is short"
264
- end
259
+ @info "the truncating logger only truncates long messages"
260
+ @info "Like this one that is this is a long and rambling message, it just keeps going and going and going, and it seems like it will never end."
261
+ @info "Not like this one, that is is short"
262
+ end
265
263
[ Info: the truncating logger only truncates long messages
266
264
[ Info: Like this one that is this is a long and rambling message, it just keeps going and going and going, and it seems like it wil...
267
265
[ Info: Not like this one, that is is short
@@ -279,7 +277,7 @@ using LoggingExtras
279
277
using Logging
280
278
281
279
function sensible_message_filter(log)
282
- length(log.message) < 1028
280
+ length(log.message) < 1028
283
281
end
284
282
285
283
global_logger(ActiveFilteredLogger(sensible_message_filter, global_logger()))
@@ -294,7 +292,7 @@ using Logging
294
292
using HTTP
295
293
296
294
function not_HTTP_message_filter(log)
297
- log._module != HTTP
295
+ log._module != HTTP
298
296
end
299
297
300
298
global_logger(EarlyFilteredLogger(not_HTTP_message_filter, global_logger()))
@@ -308,8 +306,8 @@ using Logging
308
306
using HTTP
309
307
310
308
transformer_logger(global_logger()) do log
311
- if log._module == HTTP && log.level=Logging.Debug
312
- # Merge can be used to construct a new NamedTuple
309
+ if log._module == HTTP && log.level=Logging.Debug
310
+ # Merge can be used to construct a new NamedTuple
313
311
# which effectively is the overwriting of fields of a NamedTuple
314
312
return merge(log, (; level=Logging.Info))
315
313
else
319
317
320
318
global_logger(transformer_logger)
321
319
```
322
-
323
-
0 commit comments