You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat(cache): add support for bson and yaml serializers
- Added bson and yaml serializer/deserializer support for RedisFuncCache
- Updated serializer parameter to accept a tuple or serializer name
- Added optional dependencies for bson, yaml, and other serializers
- Improved documentation for using custom serializers
- Clarified security risks associated with pickle serialization
Copy file name to clipboardExpand all lines: CHANGELOG.md
+12Lines changed: 12 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,17 @@
1
1
# Changelog
2
2
3
+
## Developing
4
+
5
+
- ✨ **New Features:**
6
+
- Added `bson` and `yaml` serializer/deserializer support for the `RedisFuncCache` class.
7
+
8
+
- 💔 **Breaking Changes:**
9
+
- The `serializer` optional parameter in the `RedisFuncCache`'s `decorate` and `__call__` methods has been replaced. It now accepts a tuple of `(serializer, deserializer)` or simply the name of the serializer function.
10
+
11
+
- 📦 **Packaging:**
12
+
- Added `bson`, `yaml` as optional dependencies, and `all` for all serializers.
13
+
- Added `types-all`, `types-PyYAML`, and `types-Pygments` as optional dependencies for typing hints.
Copy file name to clipboardExpand all lines: README.md
+38-15Lines changed: 38 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -378,7 +378,7 @@ The [`RedisFuncCache`][] instance has two arguments to control the maximum size
378
378
### Complex return types
379
379
380
380
The return value's (de)serializer is [JSON][] (`json` module of std-lib) by default, which does not work with complex objects.
381
-
However, we can still use [`pickle`][]. This can be achieved by specifying either the `serializers` argument of [`RedisFuncCache`][], or the `serializer` and `deserializer` arguments of the decorator:
381
+
However, we can still use [`pickle`][]. This can be achieved by specifying either the `serializers` argument of [`RedisFuncCache`][]'s constructor(`__init__`), or the decorator(`__call__`):
382
382
383
383
> 💡 **Example:**
384
384
>
@@ -403,19 +403,25 @@ However, we can still use [`pickle`][]. This can be achieved by specifying eithe
Other serialization functions also should be workable, such as [simplejson](https://pypi.org/project/simplejson/), [cJSON](https://github.com/DaveGamble/cJSON), [msgpack][], [YAML](https://yaml.org/), [cloudpickle](https://github.com/cloudpipe/cloudpickle), etc.
420
+
Other serialization libraries such as [bson][], [simplejson](https://pypi.org/project/simplejson/), [cJSON](https://github.com/DaveGamble/cJSON), [msgpack][], [yaml][], and [cloudpickle](https://github.com/cloudpipe/cloudpickle) are also supported.
415
421
416
-
> ⚠️ **Warning:**\
417
-
> [`pickle`][] is considered a security risk, and also cant not be used with runtime/version sensitive data. Use it cautiously and only when necessary.
418
-
>It's a good practice to only cache functions that return [JSON][]serializable simple data.
422
+
> ⚠️ **Warning:**
423
+
> The [`pickle`][] module is highly powerful but poses a significant security risk because it can execute arbitrary code during deserialization. Use it with extreme caution, especially when handling data from untrusted sources.
424
+
> For best practices, it is recommended to cache functions that return simple, [JSON][]-serializable data. If you need to serialize more complex data structures than those supported by [JSON][], consider using safer alternatives such as [bson][], [msgpack][], or [yaml][].
419
425
420
426
## Advanced Usage
421
427
@@ -425,45 +431,60 @@ The result of the decorated function is serialized by default using [JSON][] (vi
425
431
426
432
To utilize alternative serialization methods, such as [msgpack][], you have two options:
427
433
428
-
1. Specify the `serializer` argument in the constructor of [`RedisFuncCache`][], where the argument is a tuple of `(serializer, deserializer)`:
434
+
1. Specify the `serializer` argument in the constructor of [`RedisFuncCache`][], where the argument is a tuple of `(serializer, deserializer)`, or name of the serializer function:
429
435
430
436
This method applies globally: all functions decorated by this cache will use the specified serializer.
431
437
432
438
For example:
433
439
434
440
```python
435
-
import msgpack
441
+
import bson
436
442
from redis import Redis
437
443
from redis_func_cache import RedisFuncCache, LruTPolicy
438
444
445
+
def serialize(x):
446
+
return bson.encode({"return_value": x})
447
+
448
+
def deserialize(x):
449
+
return bson.decode(x)["return_value"]
450
+
439
451
cache = RedisFuncCache(
440
452
__name__,
441
453
LruTPolicy,
442
454
lambda: Redis.from_url("redis://"),
443
-
serializer=(msgpack.packb, msgpack.unpackb)
455
+
serializer=(serialize, deserialize)
444
456
)
445
457
446
458
@cache
447
-
def my_func(x):
459
+
def func():
448
460
...
449
461
```
450
462
451
-
1. Specify the `serializer` and `deserializer` arguments directly in the decorator:
463
+
1. Specify the `serializer` argument directly in the decorator. The argument should be a tuple of (`serializer`, `deserializer`) or simply the name of the serializer function.
452
464
453
465
This method applies on a per-function basis: only the decorated function will use the specified serializer.
454
466
455
467
For example:
456
468
469
+
- We can use [msgpack][] as the serializer to cache functions whose return value is binary data, which is not possible with [JSON][].
470
+
- We can use [bson][] as the serializer to cache functions whose return value is a `datetime` object, which cannot be handled by either [JSON][] or [msgpack][].
471
+
457
472
```python
458
473
import msgpack
459
474
from redis import Redis
460
475
from redis_func_cache import RedisFuncCache, LruTPolicy
@@ -729,7 +750,9 @@ docker compose up --abort-on-container-exit
729
750
[json]: https://www.json.org/ "JSON (JavaScript Object Notation) is a lightweight data-interchange format."
730
751
[`pickle`]: https://docs.python.org/library/pickle.html "The pickle module implements binary protocols for serializing and de-serializing a Python object structure."
731
752
753
+
[bson]: https://bsonspec.org/ "BSON, short for Binary JSON, is a binary-encoded serialization of JSON-like documents."
732
754
[msgpack]: https://msgpack.org/ "MessagePack is an efficient binary serialization format."
755
+
[yaml]: https://yaml.org/ "YAML is a human-friendly data serialization language for all programming languages."
0 commit comments