Skip to content

Commit 20ce46a

Browse files
authored
Add loguru support (#276)
1 parent e0c97ba commit 20ce46a

File tree

15 files changed

+672
-4
lines changed

15 files changed

+672
-4
lines changed

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@
1818
- Add test and support for Python Slim base images (#249)
1919
- Add support for the tags of Virtual Cache for Redis (#263)
2020
- Add a new configuration `kafka_namespace` to prefix the kafka topic names (#277)
21+
- Add log reporter support for loguru (#276)
2122

2223
- Plugins:
2324
- Add aioredis, aiormq, amqp, asyncpg, aio-pika, kombu RMQ plugins (#230 Missing test coverage)

docs/en/setup/Configuration.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -68,8 +68,8 @@ export SW_AGENT_YourConfiguration=YourValue
6868
| log_reporter_max_buffer_size | SW_AGENT_LOG_REPORTER_MAX_BUFFER_SIZE | <class 'int'> | 10000 | The maximum queue backlog size for sending log data to backend, logs beyond this are silently dropped. |
6969
| log_reporter_level | SW_AGENT_LOG_REPORTER_LEVEL | <class 'str'> | WARNING | This config specifies the logger levels of concern, any logs with a level below the config will be ignored. |
7070
| log_reporter_ignore_filter | SW_AGENT_LOG_REPORTER_IGNORE_FILTER | <class 'bool'> | False | This config customizes whether to ignore the application-defined logger filters, if `True`, all logs are reported disregarding any filter rules. |
71-
| log_reporter_formatted | SW_AGENT_LOG_REPORTER_FORMATTED | <class 'bool'> | True | If `True`, the log reporter will transmit the logs as formatted. Otherwise, puts logRecord.msg and logRecord.args into message content and tags(`argument.n`), respectively. Along with an `exception` tag if an exception was raised. |
72-
| log_reporter_layout | SW_AGENT_LOG_REPORTER_LAYOUT | <class 'str'> | %(asctime)s [%(threadName)s] %(levelname)s %(name)s - %(message)s | The log reporter formats the logRecord message based on the layout given. |
71+
| log_reporter_formatted | SW_AGENT_LOG_REPORTER_FORMATTED | <class 'bool'> | True | If `True`, the log reporter will transmit the logs as formatted. Otherwise, puts logRecord.msg and logRecord.args into message content and tags(`argument.n`), respectively. Along with an `exception` tag if an exception was raised. Only applies to logging module. |
72+
| log_reporter_layout | SW_AGENT_LOG_REPORTER_LAYOUT | <class 'str'> | %(asctime)s [%(threadName)s] %(levelname)s %(name)s - %(message)s | The log reporter formats the logRecord message based on the layout given. Only applies to logging module. |
7373
| cause_exception_depth | SW_AGENT_CAUSE_EXCEPTION_DEPTH | <class 'int'> | 10 | This configuration is shared by log reporter and tracer. This config limits agent to report up to `limit` stacktrace, please refer to [Python traceback]( https://docs.python.org/3/library/traceback.html#traceback.print_tb) for more explanations. |
7474
### Meter Reporter Configurations
7575
| Configuration | Environment Variable | Type | Default Value | Description |

docs/en/setup/Plugins.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ or a limitation of SkyWalking auto-instrumentation (welcome to contribute!)
3030
| [http_server](https://docs.python.org/3/library/http.server.html) | Python >=3.7 - ['*']; | `sw_http_server` |
3131
| [werkzeug](https://werkzeug.palletsprojects.com/) | Python >=3.7 - ['1.0.1', '2.0']; | `sw_http_server` |
3232
| [kafka-python](https://kafka-python.readthedocs.io) | Python >=3.7 - ['2.0']; | `sw_kafka` |
33+
| [loguru](https://pypi.org/project/loguru/) | Python >=3.7 - ['0.6.0']; | `sw_loguru` |
3334
| [mysqlclient](https://mysqlclient.readthedocs.io/) | Python >=3.7 - ['2.1.*']; | `sw_mysqlclient` |
3435
| [psycopg[binary]](https://www.psycopg.org/) | Python >=3.7 - ['3.0']; | `sw_psycopg` |
3536
| [psycopg2-binary](https://www.psycopg.org/) | Python >=3.10 - NOT SUPPORTED YET; Python >=3.7 - ['2.9']; | `sw_psycopg2` |

docs/en/setup/advanced/LogReporter.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Python Agent Log Reporter
22

3-
This functionality reports logs collected from the Python logging module (in theory, also logging libraries depending on the core logging module).
3+
This functionality reports logs collected from the Python logging module (in theory, also logging libraries depending on the core logging module) and loguru module.
44

55
To utilize this feature, you will need to add some new configurations to the agent initialization step.
66

poetry.lock

Lines changed: 35 additions & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

pyproject.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -126,6 +126,7 @@ aiormq = "^6.4.2"
126126
asyncpg = "^0.27.0"
127127
happybase = "1.2.0"
128128
websockets = "^10.4"
129+
loguru = "^0.6.0"
129130

130131
[tool.poetry.group.lint.dependencies]
131132
flake8 = "^5.0.4"

skywalking/config.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -146,8 +146,10 @@
146146
log_reporter_ignore_filter: bool = os.getenv('SW_AGENT_LOG_REPORTER_IGNORE_FILTER', '').lower() == 'true'
147147
# If `True`, the log reporter will transmit the logs as formatted. Otherwise, puts logRecord.msg and logRecord.args
148148
# into message content and tags(`argument.n`), respectively. Along with an `exception` tag if an exception was raised.
149+
# Only applies to logging module.
149150
log_reporter_formatted: bool = os.getenv('SW_AGENT_LOG_REPORTER_FORMATTED', '').lower() != 'false'
150151
# The log reporter formats the logRecord message based on the layout given.
152+
# Only applies to logging module.
151153
log_reporter_layout: str = os.getenv('SW_AGENT_LOG_REPORTER_LAYOUT',
152154
'%(asctime)s [%(threadName)s] %(levelname)s %(name)s - %(message)s')
153155
# This configuration is shared by log reporter and tracer.

skywalking/plugins/sw_loguru.py

Lines changed: 227 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,227 @@
1+
#
2+
# Licensed to the Apache Software Foundation (ASF) under one or more
3+
# contributor license agreements. See the NOTICE file distributed with
4+
# this work for additional information regarding copyright ownership.
5+
# The ASF licenses this file to You under the Apache License, Version 2.0
6+
# (the "License"); you may not use this file except in compliance with
7+
# the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
#
17+
18+
import logging
19+
import sys
20+
import traceback
21+
from multiprocessing import current_process
22+
from os.path import basename, splitext
23+
from threading import current_thread
24+
25+
from skywalking import config, agent
26+
from skywalking.protocol.common.Common_pb2 import KeyStringValuePair
27+
from skywalking.protocol.logging.Logging_pb2 import LogData, LogDataBody, TraceContext, LogTags, TextLog
28+
from skywalking.trace.context import get_context
29+
from skywalking.utils.exception import IllegalStateError
30+
from skywalking.utils.filter import sw_filter
31+
32+
link_vector = ['https://pypi.org/project/loguru/']
33+
support_matrix = {
34+
'loguru': {
35+
'>=3.7': ['0.6.0']
36+
}
37+
}
38+
note = """"""
39+
40+
41+
def install():
42+
from loguru import logger
43+
from loguru._recattrs import RecordException, RecordFile, RecordLevel, RecordProcess, RecordThread
44+
from loguru._datetime import aware_now
45+
from loguru._get_frame import get_frame
46+
from loguru._logger import start_time, context as logger_context, Logger
47+
from types import MethodType
48+
49+
_log = logger._log
50+
log_reporter_level = logging.getLevelName(config.log_reporter_level) # type: int
51+
52+
def gen_record(self, level_id, static_level_no, from_decorator, options, message, args, kwargs):
53+
""" Generate log record as loguru.logger._log """
54+
core = self._core
55+
56+
if not core.handlers:
57+
return
58+
59+
(exception, depth, record, lazy, colors, raw, capture, patcher, extra) = options
60+
61+
frame = get_frame(depth + 2)
62+
63+
try:
64+
name = frame.f_globals['__name__']
65+
except KeyError:
66+
name = None
67+
68+
try:
69+
if not core.enabled[name]:
70+
return
71+
except KeyError:
72+
enabled = core.enabled
73+
if name is None:
74+
status = core.activation_none
75+
enabled[name] = status
76+
if not status:
77+
return
78+
else:
79+
dotted_name = name + '.'
80+
for dotted_module_name, status in core.activation_list:
81+
if dotted_name[: len(dotted_module_name)] == dotted_module_name:
82+
if status:
83+
break
84+
enabled[name] = False
85+
return
86+
enabled[name] = True
87+
88+
current_datetime = aware_now()
89+
90+
if level_id is None:
91+
level_icon = ' '
92+
level_no = static_level_no
93+
level_name = f'Level {level_no}' # not really level name, just as loguru
94+
else:
95+
level_name, level_no, _, level_icon = core.levels[level_id]
96+
97+
if level_no < core.min_level:
98+
return
99+
100+
code = frame.f_code
101+
file_path = code.co_filename
102+
file_name = basename(file_path)
103+
thread = current_thread()
104+
process = current_process()
105+
elapsed = current_datetime - start_time
106+
107+
if exception:
108+
if isinstance(exception, BaseException):
109+
type_, value, traceback = (type(exception), exception, exception.__traceback__)
110+
elif isinstance(exception, tuple):
111+
type_, value, traceback = exception
112+
else:
113+
type_, value, traceback = sys.exc_info()
114+
exception = RecordException(type_, value, traceback)
115+
else:
116+
exception = None
117+
118+
log_record = {
119+
'elapsed': elapsed,
120+
'exception': exception,
121+
'extra': {**core.extra, **logger_context.get(), **extra},
122+
'file': RecordFile(file_name, file_path),
123+
'function': code.co_name,
124+
'level': RecordLevel(level_name, level_no, level_icon),
125+
'line': frame.f_lineno,
126+
'message': str(message),
127+
'module': splitext(file_name)[0],
128+
'name': name,
129+
'process': RecordProcess(process.ident, process.name),
130+
'thread': RecordThread(thread.ident, thread.name),
131+
'time': current_datetime,
132+
}
133+
134+
if capture and kwargs:
135+
log_record['extra'].update(kwargs)
136+
137+
if record:
138+
kwargs.update(record=log_record)
139+
140+
if args or kwargs:
141+
log_record['message'] = message.format(*args, **kwargs)
142+
143+
if core.patcher:
144+
core.patcher(log_record)
145+
146+
if patcher:
147+
patcher(log_record)
148+
149+
return log_record
150+
151+
def _sw_log(self, level_id, static_level_no, from_decorator, options, message, args, kwargs):
152+
_log(level_id, static_level_no, from_decorator, options, message, args, kwargs)
153+
record = gen_record(self, level_id, static_level_no, from_decorator, options, message, args, kwargs)
154+
if record is None:
155+
return
156+
157+
core = self._core
158+
159+
if record['level'].no < log_reporter_level:
160+
return
161+
162+
if not config.log_reporter_ignore_filter and record['level'].no < core.min_level: # ignore filtered logs
163+
return
164+
165+
# loguru has only one logger. Use tags referring Python-Agent doc
166+
core_tags = [
167+
KeyStringValuePair(key='level', value=record['level'].name),
168+
KeyStringValuePair(key='logger', value='loguru'),
169+
KeyStringValuePair(key='thread', value=record['thread'].name),
170+
]
171+
tags = LogTags()
172+
tags.data.extend(core_tags)
173+
174+
exception = record['exception']
175+
if exception:
176+
stack_trace = ''.join(traceback.format_exception(exception.type, exception.value, exception.traceback,
177+
limit=config.cause_exception_depth))
178+
tags.data.append(KeyStringValuePair(key='exception',
179+
value=sw_filter(stack_trace)
180+
)) # \n doesn't work in tags for UI
181+
182+
context = get_context()
183+
184+
active_span_id = -1
185+
primary_endpoint_name = ''
186+
187+
try:
188+
# Try to extract active span, if user code/plugin code throws uncaught
189+
# exceptions before any span is even created, just ignore these fields and
190+
# avoid appending 'no active span' traceback that could be confusing.
191+
# Or simply the log is generated outside any span context.
192+
active_span_id = context.active_span.sid
193+
primary_endpoint_name = context.primary_endpoint.get_name()
194+
except IllegalStateError:
195+
pass
196+
197+
log_data = LogData(
198+
timestamp=round(record['time'].timestamp() * 1000),
199+
service=config.service_name,
200+
serviceInstance=config.service_instance,
201+
body=LogDataBody(
202+
type='text',
203+
text=TextLog(
204+
text=sw_filter(message)
205+
)
206+
),
207+
tags=tags,
208+
)
209+
210+
if active_span_id != -1:
211+
trace_context = TraceContext(
212+
traceId=str(context.segment.related_traces[0]),
213+
traceSegmentId=str(context.segment.segment_id),
214+
spanId=active_span_id
215+
)
216+
log_data.traceContext.CopyFrom(trace_context)
217+
218+
if primary_endpoint_name:
219+
log_data.endpoint = primary_endpoint_name
220+
221+
agent.archive_log(log_data)
222+
223+
# Bind _sw_log function to default logger instance.
224+
bound_sw_log = MethodType(_sw_log, logger)
225+
logger._log = bound_sw_log
226+
# Bind _sw_log function to Logger class for new instance.
227+
Logger._log = _sw_log
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
#
2+
# Licensed to the Apache Software Foundation (ASF) under one or more
3+
# contributor license agreements. See the NOTICE file distributed with
4+
# this work for additional information regarding copyright ownership.
5+
# The ASF licenses this file to You under the Apache License, Version 2.0
6+
# (the "License"); you may not use this file except in compliance with
7+
# the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
#

0 commit comments

Comments
 (0)