Skip to content

Commit 98b41e2

Browse files
authored
Merge pull request #87 from akiva-skolnik/master
Fixed DeprecationWarning & improved code format
2 parents 73adf14 + a464412 commit 98b41e2

File tree

16 files changed

+154
-107
lines changed

16 files changed

+154
-107
lines changed

.github/workflows/auto-release.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
name: Pypi Release
22
on:
33
workflow_run:
4-
workflows: ["CI Tests"]
5-
branches: [master]
4+
workflows: [ "CI Tests" ]
5+
branches: [ master ]
66
types:
77
- completed
88
# Allows you to run this workflow manually from the Actions tab

.github/workflows/tests.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
name: CI Tests
2-
on: [push]
2+
on: [ push ]
33
jobs:
44
run-tests:
55
runs-on: ubuntu-latest

README.md

Lines changed: 114 additions & 76 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,9 @@
66

77
### Deprecation announcement
88

9-
Version 3.0.0 of this project ends support for Python 2.7, 3.3, and 3.4. We recommend migrating your projects to Python 3.5 or newer as soon as possible. We'll be happy to answer any questions you have in [a GitHub issue](https://github.com/logzio/logzio-python-handler/issues).
9+
Version 3.0.0 of this project ends support for Python 2.7, 3.3, and 3.4. We recommend migrating your projects to Python
10+
3.5 or newer as soon as possible. We'll be happy to answer any questions you have
11+
in [a GitHub issue](https://github.com/logzio/logzio-python-handler/issues).
1012

1113
Thanks! <br>
1214
The Logz.io Integrations team
@@ -15,31 +17,38 @@ The Logz.io Integrations team
1517

1618
This is a Python handler that sends logs in bulk over HTTPS to Logz.io.
1719
The handler uses a subclass named LogzioSender (which can be used without this handler as well, to ship raw data).
18-
The LogzioSender class opens a new Thread, that consumes from the logs queue. Each iteration (its frequency of which can be configured by the logs_drain_timeout parameter), will try to consume the queue in its entirety.
20+
The LogzioSender class opens a new Thread, that consumes from the logs queue. Each iteration (its frequency of which can
21+
be configured by the logs_drain_timeout parameter), will try to consume the queue in its entirety.
1922
Logs will get divided into separate bulks, based on their size.
20-
LogzioSender will check if the main thread is alive. In case the main thread quits, it will try to consume the queue one last time, and then exit. So your program can hang for a few seconds, until the logs are drained.
21-
In case the logs failed to be sent to Logz.io after a couple of tries, they will be written to the local file system. You can later upload them to Logz.io using curl.
22-
23+
LogzioSender will check if the main thread is alive. In case the main thread quits, it will try to consume the queue one
24+
last time, and then exit. So your program can hang for a few seconds, until the logs are drained.
25+
In case the logs failed to be sent to Logz.io after a couple of tries, they will be written to the local file system.
26+
You can later upload them to Logz.io using curl.
2327

2428
## Installation
29+
2530
```bash
2631
pip install logzio-python-handler
2732
```
2833

29-
If you'd like to use [Trace context](#trace-context), you need to install the OpenTelemetry logging instrumentation dependency by running the following command:
34+
If you'd like to use [Trace context](#trace-context), you need to install the OpenTelemetry logging instrumentation
35+
dependency by running the following command:
3036

3137
```bash
3238
pip install logzio-python-handler[opentelemetry-logging]
3339
```
40+
3441
## Tested Python Versions
42+
3543
Travis CI will build this handler and test against:
36-
- "3.5"
37-
- "3.6"
38-
- "3.7"
39-
- "3.8"
40-
- "3.9"
41-
- "3.10"
42-
- "3.11"
44+
45+
- "3.5"
46+
- "3.6"
47+
- "3.7"
48+
- "3.8"
49+
- "3.9"
50+
- "3.10"
51+
- "3.11"
4352

4453
We can't ensure compatibility to any other version, as we can't test it automatically.
4554

@@ -53,46 +62,56 @@ $ tox
5362
```
5463

5564
## Python configuration
65+
5666
#### Config File
67+
5768
```python
5869
[handlers]
59-
keys=LogzioHandler
70+
keys = LogzioHandler
6071

6172
[handler_LogzioHandler]
62-
class=logzio.handler.LogzioHandler
63-
formatter=logzioFormat
73+
74+
75+
class =logzio.handler.LogzioHandler
76+
77+
78+
formatter = logzioFormat
6479

6580
# Parameters must be set in order. Replace these parameters with your configuration.
66-
args=('<<LOG-SHIPPING-TOKEN>>', '<<LOG-TYPE>>', <<TIMEOUT>>, 'https://<<LISTENER-HOST>>:8071', <<DEBUG-FLAG>>,<<NETWORKING-TIMEOUT>>,<<RETRY-LIMIT>>,<<RETRY-TIMEOUT>>)
81+
args = ('<<LOG-SHIPPING-TOKEN>>', '<<LOG-TYPE>>',
82+
<< TIMEOUT >>, 'https://<<LISTENER-HOST>>:8071', << DEBUG-FLAG >>, << NETWORKING-TIMEOUT >>, << RETRY-LIMIT >>, << RETRY-TIMEOUT >>)
6783

6884
[formatters]
69-
keys=logzioFormat
85+
keys = logzioFormat
7086

7187
[loggers]
72-
keys=root
88+
keys = root
7389

7490
[logger_root]
75-
handlers=LogzioHandler
76-
level=INFO
91+
handlers = LogzioHandler
92+
level = INFO
7793

7894
[formatter_logzioFormat]
79-
format={"additional_field": "value"}
95+
format = {"additional_field": "value"}
8096
```
97+
8198
*args=() arguments, by order*
82-
- Your logz.io token
83-
- Log type, for searching in logz.io (defaults to "python")
84-
- Time to sleep between draining attempts (defaults to "3")
85-
- Logz.io Listener address (defaults to "https://listener.logz.io:8071")
86-
- Debug flag. Set to True, will print debug messages to stdout. (defaults to "False")
87-
- Backup logs flag. Set to False, will disable the local backup of logs in case of failure. (defaults to "True")
88-
- Network timeout, in seconds, int or float, for sending the logs to logz.io. (defaults to 10)
89-
- Retries number (retry_no, defaults to 4).
90-
- Retry timeout (retry_timeout) in seconds (defaults to 2).
91-
92-
Please note, that you have to configure those parameters by this exact order.
93-
i.e. you cannot set Debug to true, without configuring all of the previous parameters as well.
99+
100+
- Your logz.io token
101+
- Log type, for searching in logz.io (defaults to "python")
102+
- Time to sleep between draining attempts (defaults to "3")
103+
- Logz.io Listener address (defaults to "https://listener.logz.io:8071")
104+
- Debug flag. Set to True, will print debug messages to stdout. (defaults to "False")
105+
- Backup logs flag. Set to False, will disable the local backup of logs in case of failure. (defaults to "True")
106+
- Network timeout, in seconds, int or float, for sending the logs to logz.io. (defaults to 10)
107+
- Retries number (retry_no, defaults to 4).
108+
- Retry timeout (retry_timeout) in seconds (defaults to 2).
109+
110+
Please note, that you have to configure those parameters by this exact order.
111+
i.e. you cannot set Debug to true, without configuring all of the previous parameters as well.
94112

95113
#### Dict Config
114+
96115
```python
97116
LOGGING = {
98117
'version': 1,
@@ -125,24 +144,33 @@ LOGGING = {
125144
}
126145
}
127146
```
147+
128148
Replace:
149+
129150
* <<LOGZIO-TOKEN>> - your logz.io account token.
130-
* <<LOGZIO-URL>> - logz.io url, as described [here](https://docs.logz.io/user-guide/accounts/account-region.html#regions-and-urls).
151+
* <<LOGZIO-URL>> - logz.io url, as
152+
described [here](https://docs.logz.io/user-guide/accounts/account-region.html#regions-and-urls).
153+
131154
#### Serverless platforms
132155

133-
If you're using a serverless function, you'll need to import and add the LogzioFlusher annotation before your sender function. To do this, in the code sample below, uncomment the `import` statement and the `@LogzioFlusher(logger)` annotation line.
134-
**Note:** For the LogzioFlusher to work properly, you'll need to make sure that the Logz.io. handler is added to the root logger. See the configuration above for an example.
156+
If you're using a serverless function, you'll need to import and add the LogzioFlusher annotation before your sender
157+
function. To do this, in the code sample below, uncomment the `import` statement and the `@LogzioFlusher(logger)`
158+
annotation line.
159+
**Note:** For the LogzioFlusher to work properly, you'll need to make sure that the Logz.io. handler is added to the
160+
root logger. See the configuration above for an example.
135161

136162
#### Dynamic Extra Fields
163+
137164
If you prefer, you can add extra fields to your logs dynamically, and not pre-defining them in the configuration.
138165
This way, you can allow different logs to have different extra fields.
139-
Example in the code below.
166+
Example in the code below.
140167

141168
#### Code Example
142169

143170
```python
144171
import logging
145172
import logging.config
173+
146174
# If you're using a serverless function, uncomment.
147175
# from logzio.flusher import LogzioFlusher
148176

@@ -153,55 +181,62 @@ import logging.config
153181
logging.config.dictConfig(LOGGING)
154182
logger = logging.getLogger('superAwesomeLogzioLogger')
155183

184+
156185
# If you're using a serverless function, uncomment.
157186
# @LogzioFlusher(logger)
158187
def my_func():
159188
logger.info('Test log')
160-
logger.warn('Warning')
189+
logger.warning('Warning')
161190

162191
try:
163-
1/0
192+
1 / 0
164193
except:
165194
logger.exception("Supporting exceptions too!")
166195

167-
# Example additional code that demonstrates how to dynamically add/remove fields within the code, make sure class is imported.
196+
# Example additional code that demonstrates how to dynamically add/remove fields within the code, make sure class is imported.
168197

169198
logger.info("Test log") # Outputs: {"message":"Test log"}
170-
171-
extra_fields = {"foo":"bar","counter":1}
199+
200+
extra_fields = {"foo": "bar", "counter": 1}
172201
logger.addFilter(ExtraFieldsLogFilter(extra_fields))
173202
logger.warning("Warning test log") # Outputs: {"message":"Warning test log","foo":"bar","counter":1}
174-
175-
error_fields = {"err_msg":"Failed to run due to exception.","status_code":500}
203+
204+
error_fields = {"err_msg": "Failed to run due to exception.", "status_code": 500}
176205
logger.addFilter(ExtraFieldsLogFilter(error_fields))
177-
logger.error("Error test log") # Outputs: {"message":"Error test log","foo":"bar","counter":1,"err_msg":"Failed to run due to exception.","status_code":500}
178-
206+
logger.error(
207+
"Error test log") # Outputs: {"message":"Error test log","foo":"bar","counter":1,"err_msg":"Failed to run due to exception.","status_code":500}
208+
179209
# If you'd like to remove filters from future logs using the logger.removeFilter option:
180210
logger.removeFilter(ExtraFieldsLogFilter(error_fields))
181-
logger.debug("Debug test log") # Outputs: {"message":"Debug test log","foo":"bar","counter":1}
211+
logger.debug("Debug test log") # Outputs: {"message":"Debug test log","foo":"bar","counter":1}
182212

183213
```
184214

185215
#### Extra Fields
186-
In case you need to dynamic metadata to a speific log and not [dynamically to the logger](#dynamic-extra-fields), other than the constant metadata from the formatter, you can use the "extra" parameter.
187-
All key values in the dictionary passed in "extra" will be presented in Logz.io as new fields in the log you are sending.
216+
217+
In case you need to dynamic metadata to a speific log and not [dynamically to the logger](#dynamic-extra-fields), other
218+
than the constant metadata from the formatter, you can use the "extra" parameter.
219+
All key values in the dictionary passed in "extra" will be presented in Logz.io as new fields in the log you are
220+
sending.
188221
Please note, that you cannot override default fields by the python logger (i.e. lineno, thread, etc..)
189222
For example:
190223

191224
```python
192-
logger.info('Warning', extra={'extra_key':'extra_value'})
225+
logger.info('Warning', extra={'extra_key': 'extra_value'})
193226
```
194227

195228
#### Trace context
196229

197-
If you're sending traces with OpenTelemetry instrumentation (auto or manual), you can correlate your logs with the trace context.
230+
If you're sending traces with OpenTelemetry instrumentation (auto or manual), you can correlate your logs with the trace
231+
context.
198232
That way, your logs will have traces data in it, such as service name, span id and trace id.
199233

200234
Make sure to install the OpenTelemetry logging instrumentation dependecy by running the following command:
201235

202236
```shell
203237
pip install logzio-python-handler[opentelemetry-logging]
204238
```
239+
205240
To enable this feature, set the `add_context` param in your handler configuration to `True`, like in this example:
206241

207242
```python
@@ -239,6 +274,7 @@ LOGGING = {
239274
```
240275

241276
#### Django configuration
277+
242278
```python
243279
LOGGING = {
244280
'version': 1,
@@ -284,49 +320,50 @@ LOGGING = {
284320

285321
```
286322

287-
288323
## Release Notes
324+
325+
- 4.1.2
326+
- Fix DeprecationWarning
327+
- Adjusted tests to logging:3.12
289328
- 4.1.1
290-
- Fix flusher issues with serverless
329+
- Fix flusher issues with serverless
291330
- 4.1.0
292-
- Add ability to dynamically attach extra fields to the logs.
293-
- Import opentelemetry logging dependency only if trace context is enabled and dependency is installed manually.
294-
- Updated `opentelemetry-instrumentation-logging==0.39b0`
295-
- Updated `setuptools>=68.0.0`
296-
- Added tests for Python versions: 3.9, 3.10, 3.11
331+
- Add ability to dynamically attach extra fields to the logs.
332+
- Import opentelemetry logging dependency only if trace context is enabled and dependency is installed manually.
333+
- Updated `opentelemetry-instrumentation-logging==0.39b0`
334+
- Updated `setuptools>=68.0.0`
335+
- Added tests for Python versions: 3.9, 3.10, 3.11
297336
- 4.0.2
298-
- Fix bug for logging exceptions ([#76](https://github.com/logzio/logzio-python-handler/pull/76))
337+
- Fix bug for logging exceptions ([#76](https://github.com/logzio/logzio-python-handler/pull/76))
299338
- 4.0.1
300-
- Updated `protobuf>=3.20.2`.
301-
- Added dependency `setuptools>=65.5.1`
302-
303-
- 4.0.0
304-
- Add ability to automatically attach trace context to the logs.
339+
- Updated `protobuf>=3.20.2`.
340+
- Added dependency `setuptools>=65.5.1`
305341

342+
- 4.0.0
343+
- Add ability to automatically attach trace context to the logs.
306344

307345
<details>
308346
<summary markdown="span"> Expand to check old versions </summary>
309347

310-
311348
- 3.1.1
312-
- Bug fixes (issue #68, exception message formatting)
313-
- Added CI: Tests and Auto release
314-
349+
- Bug fixes (issue #68, exception message formatting)
350+
- Added CI: Tests and Auto release
351+
315352
- 3.1.0
316353
- Bug fixes
317354
- Retry number and timeout is now configurable
318-
355+
319356
- 3.0.0
320357
- Deprecated `python2.7` & `python3.4`
321358
- Changed log levels on `_flush_queue()` method (@hilsenrat)
322359

323360
- 2.0.15
324361
- Added flusher decorator for serverless platforms(@mcmasty)
325-
- Add support for `python3.7` and `python3.8`
362+
- Add support for `python3.7` and `python3.8`
326363

327-
- 2.0.13
364+
- 2.0.13
328365
- Add support for `pypy` and `pypy3`(@rudaporto-olx)
329-
- Add timeout for requests.post() (@oseemann)
366+
- Add timeout for requests.post() (@oseemann)
330367
- 2.0.12 - Support disable logs local backup
331368
- 2.0.11 - Completely isolate exception from the message
332369
- 2.0.10 - Not ignoring formatting on exceptions
@@ -340,11 +377,12 @@ LOGGING = {
340377
- 2.0.2 - Support for formatted messages (Thanks @johnraz!)
341378
- 2.0.1 - Added __all__ to __init__.py, so support * imports
342379
- 2.0.0 - Production, stable release.
343-
- *BREAKING* - Configuration option logs_drain_count was removed, and the order of the parameters has changed for better simplicity. Please review the parameters section above.
344-
- Introducing the LogzioSender class, which is generic and can be used without the handler wrap to ship raw data to Logz.io. Just create a new instance of the class, and use the append() method.
380+
- *BREAKING* - Configuration option logs_drain_count was removed, and the order of the parameters has changed for
381+
better simplicity. Please review the parameters section above.
382+
- Introducing the LogzioSender class, which is generic and can be used without the handler wrap to ship raw data to
383+
Logz.io. Just create a new instance of the class, and use the append() method.
345384
- Simplifications and Robustness
346385
- Full testing framework
347386
- 1.X - Beta versions
348-
349387

350388
</details>

logzio/flusher.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,4 +17,5 @@ def wrapper(*args, **kwargs):
1717
raise
1818
finally:
1919
[h.flush() for h in self.logger.handlers]
20+
2021
return wrapper

0 commit comments

Comments
 (0)