Skip to content

Commit f941604

Browse files
committed
feat(cache): add support for bson and yaml serializers
- Added bson and yaml serializer/deserializer support for RedisFuncCache - Updated serializer parameter to accept a tuple or serializer name - Added optional dependencies for bson, yaml, and other serializers - Improved documentation for using custom serializers - Clarified security risks associated with pickle serialization
1 parent a020bec commit f941604

File tree

2 files changed

+50
-15
lines changed

2 files changed

+50
-15
lines changed

CHANGELOG.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,17 @@
11
# Changelog
22

3+
## Developing
4+
5+
-**New Features:**
6+
- Added `bson` and `yaml` serializer/deserializer support for the `RedisFuncCache` class.
7+
8+
- 💔 **Breaking Changes:**
9+
- The `serializer` optional parameter in the `RedisFuncCache`'s `decorate` and `__call__` methods has been replaced. It now accepts a tuple of `(serializer, deserializer)` or simply the name of the serializer function.
10+
11+
- 📦 **Packaging:**
12+
- Added `bson`, `yaml` as optional dependencies, and `all` for all serializers.
13+
- Added `types-all`, `types-PyYAML`, and `types-Pygments` as optional dependencies for typing hints.
14+
315
## v0.3
416

517
> 📅 2025-01-08

README.md

Lines changed: 38 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -378,7 +378,7 @@ The [`RedisFuncCache`][] instance has two arguments to control the maximum size
378378
### Complex return types
379379
380380
The return value's (de)serializer is [JSON][] (`json` module of std-lib) by default, which does not work with complex objects.
381-
However, we can still use [`pickle`][]. This can be achieved by specifying either the `serializers` argument of [`RedisFuncCache`][], or the `serializer` and `deserializer` arguments of the decorator:
381+
However, we can still use [`pickle`][]. This can be achieved by specifying either the `serializers` argument of [`RedisFuncCache`][]'s constructor(`__init__`), or the decorator(`__call__`):
382382
383383
> 💡 **Example:**
384384
>
@@ -403,19 +403,25 @@ However, we can still use [`pickle`][]. This can be achieved by specifying eithe
403403
> serializer=(pickle.dumps, pickle.loads)
404404
> )
405405
>
406-
> # or just like this:
406+
> # or like this:
407407
> cache = RedisFuncCache(__name__, LruTPolicy, lambda: Redis.from_url("redis://"))
408408
>
409-
> @cache(serializer=pickle.loads, deserializer=pickle.dumps)
409+
> @cache(serializer=pickle.loads, pickle.dumps)
410410
> def my_func_with_complex_return_value(x):
411411
> ...
412+
>
413+
> # or just like this:
414+
> @cache(serializer="pickle")
415+
> def my_func_with_complex_return_value(x):
416+
> ...
417+
>
412418
> ```
413419
414-
Other serialization functions also should be workable, such as [simplejson](https://pypi.org/project/simplejson/), [cJSON](https://github.com/DaveGamble/cJSON), [msgpack][], [YAML](https://yaml.org/), [cloudpickle](https://github.com/cloudpipe/cloudpickle), etc.
420+
Other serialization libraries such as [bson][], [simplejson](https://pypi.org/project/simplejson/), [cJSON](https://github.com/DaveGamble/cJSON), [msgpack][], [yaml][], and [cloudpickle](https://github.com/cloudpipe/cloudpickle) are also supported.
415421
416-
> ⚠️ **Warning:**\
417-
> [`pickle`][] is considered a security risk, and also cant not be used with runtime/version sensitive data. Use it cautiously and only when necessary.
418-
> It's a good practice to only cache functions that return [JSON][] serializable simple data.
422+
> ⚠️ **Warning:**
423+
> The [`pickle`][] module is highly powerful but poses a significant security risk because it can execute arbitrary code during deserialization. Use it with extreme caution, especially when handling data from untrusted sources.
424+
> For best practices, it is recommended to cache functions that return simple, [JSON][]-serializable data. If you need to serialize more complex data structures than those supported by [JSON][], consider using safer alternatives such as [bson][], [msgpack][], or [yaml][].
419425
420426
## Advanced Usage
421427
@@ -425,45 +431,60 @@ The result of the decorated function is serialized by default using [JSON][] (vi
425431
426432
To utilize alternative serialization methods, such as [msgpack][], you have two options:
427433
428-
1. Specify the `serializer` argument in the constructor of [`RedisFuncCache`][], where the argument is a tuple of `(serializer, deserializer)`:
434+
1. Specify the `serializer` argument in the constructor of [`RedisFuncCache`][], where the argument is a tuple of `(serializer, deserializer)`, or name of the serializer function:
429435
430436
This method applies globally: all functions decorated by this cache will use the specified serializer.
431437
432438
For example:
433439
434440
```python
435-
import msgpack
441+
import bson
436442
from redis import Redis
437443
from redis_func_cache import RedisFuncCache, LruTPolicy
438444
445+
def serialize(x):
446+
return bson.encode({"return_value": x})
447+
448+
def deserialize(x):
449+
return bson.decode(x)["return_value"]
450+
439451
cache = RedisFuncCache(
440452
__name__,
441453
LruTPolicy,
442454
lambda: Redis.from_url("redis://"),
443-
serializer=(msgpack.packb, msgpack.unpackb)
455+
serializer=(serialize, deserialize)
444456
)
445457
446458
@cache
447-
def my_func(x):
459+
def func():
448460
...
449461
```
450462
451-
1. Specify the `serializer` and `deserializer` arguments directly in the decorator:
463+
1. Specify the `serializer` argument directly in the decorator. The argument should be a tuple of (`serializer`, `deserializer`) or simply the name of the serializer function.
452464
453465
This method applies on a per-function basis: only the decorated function will use the specified serializer.
454466
455467
For example:
456468
469+
- We can use [msgpack][] as the serializer to cache functions whose return value is binary data, which is not possible with [JSON][].
470+
- We can use [bson][] as the serializer to cache functions whose return value is a `datetime` object, which cannot be handled by either [JSON][] or [msgpack][].
471+
457472
```python
458473
import msgpack
459474
from redis import Redis
460475
from redis_func_cache import RedisFuncCache, LruTPolicy
461476
462477
cache = RedisFuncCache(__name__, LruTPolicy, lambda: Redis.from_url("redis://"))
463478
464-
@cache(serializer=msgpack.packb, deserializer=msgpack.unpackb)
465-
def my_func(x):
466-
...
479+
@cache(serializer=(msgpack.packb, msgpack.unpackb))
480+
def create_or_get_token(user: str) -> bytes:
481+
from secrets import token_bytes
482+
return token_bytes(32)
483+
484+
@cache(serializer="bson")
485+
def now_time():
486+
from datetime import datetime
487+
return datetime.now()
467488
```
468489
469490
### Custom key format
@@ -729,7 +750,9 @@ docker compose up --abort-on-container-exit
729750
[json]: https://www.json.org/ "JSON (JavaScript Object Notation) is a lightweight data-interchange format."
730751
[`pickle`]: https://docs.python.org/library/pickle.html "The pickle module implements binary protocols for serializing and de-serializing a Python object structure."
731752
753+
[bson]: https://bsonspec.org/ "BSON, short for Bin­ary JSON, is a bin­ary-en­coded seri­al­iz­a­tion of JSON-like doc­u­ments."
732754
[msgpack]: https://msgpack.org/ "MessagePack is an efficient binary serialization format."
755+
[yaml]: https://yaml.org/ "YAML is a human-friendly data serialization language for all programming languages."
733756
734757
[`RedisFuncCache`]: redis_func_cache.cache.RedisFuncCache
735758
[`AbstractPolicy`]: redis_func_cache.policies.abstract.AbstractPolicy

0 commit comments

Comments
 (0)