Add initial support for crypto callbacks.#114
Conversation
JeremiahM37
left a comment
There was a problem hiding this comment.
Skoll Code Review
Scan type: review
Overall recommendation: REQUEST_CHANGES
Findings: 14 total — 9 posted, 5 skipped
7 finding(s) posted as inline comments (see file-level comments below)
Posted findings
- [High] Importing wolfcrypt fails when CRYPTO_CB is disabled at build time —
wolfcrypt/__init__.py:49 and wolfcrypt/cryptocb.py:33-58 - [High] Double unregister of device on context-manager exit + GC —
wolfcrypt/cryptocb.py:73-77, 117-118 - [Medium] Return value of wolfCrypt_Init not checked —
wolfcrypt/__init__.py:52 - [Medium] Debug print() statements left in production callback dispatcher —
wolfcrypt/cryptocb.py:80, 84, 85, 107, 111 - [Medium] Hash digest dict KeyError on unmapped hash types —
wolfcrypt/cryptocb.py:84, 90 - [Medium] Digest output buffer not size-validated —
wolfcrypt/cryptocb.py:90 - [Medium] RNG output length not validated —
wolfcrypt/cryptocb.py:96-97 - [Medium] No tests for hash callbacks despite PR claim —
tests/test_cryptocb.py - [Low] Free-floating string literal used as 'comment' in build_ffi.py is misleading —
scripts/build_ffi.py:1376-1412
Skipped findings
- [Medium]
Anonymous-union cdef relies on '...' partial layout — may not match reality - [Low]
ctx==NULL fallback would never be reached — wrong return code - [Low]
Type annotations on overridable methods declare positional args with leading underscores - [Low]
rng_callback signature passes raw cffi pointer with no type annotation - [Info]
Stale exception import in init.py guard
Review generated by Skoll
|
Hi @roberthdevries nice work on this. See peer review feedback and also fix merge conflicts. Thanks |
e10a1d3 to
1f974ad
Compare
dgarske
left a comment
There was a problem hiding this comment.
Skoll Code Review
Scan type: reviewOverall recommendation: REQUEST_CHANGES
Findings: 7 total — 7 posted, 0 skipped
6 finding(s) posted as inline comments (see file-level comments below)
Posted findings
- [High] Callback exceptions are converted to 0 (success) with unwritten output buffer —
wolfcrypt/__init__.py:57-63, wolfcrypt/cryptocb.py:103-138 - [Medium] cipher_callback path returns success (0) without performing any cipher operation —
wolfcrypt/cryptocb.py:125-127,149-150 - [Medium] Hash callbacks and error paths are untested; default-device-id test asserts nothing —
tests/test_cryptocb.py:33-53 - [Medium] del unregister not guarded against interpreter-shutdown errors —
wolfcrypt/cryptocb.py:100-101,152-153 - [Medium] CRYPTO_CB cdef hash union depends on SHA/SHA256/SHA384/SHA512/SHA3 cdefs being present —
scripts/build_ffi.py:1424-1440 - [Low] Inconsistent logging style in callback (f-string vs lazy %-formatting) —
wolfcrypt/cryptocb.py:104,109 - [Low] wolfCrypt_Init() added with no corresponding cleanup —
wolfcrypt/__init__.py:53-55
Review generated by Skoll
| ) | ||
| _ffi.buffer(info.hash.digest, DIGEST_SIZE[info.hash.type])[:] = digest | ||
| return 0 | ||
| if info.algo_type == _lib.WC_ALGO_TYPE_CIPHER: |
There was a problem hiding this comment.
🟠 [Medium] cipher_callback path returns success (0) without performing any cipher operation
For WC_ALGO_TYPE_CIPHER, callback() calls self.cipher_callback(device_id) and then unconditionally return 0. The cipher fields of wc_CryptoInfo are intentionally not exposed in the cdef (commented out per the PR description), so a subclass overriding cipher_callback has no way to read inputs or write outputs, yet a no-op override would cause wolfCrypt to be told the cipher succeeded while no encryption/decryption was actually performed. Out of the box this is masked because the default cipher_callback raises NotImplementedError (caught -> CRYPTOCB_UNAVAILABLE), but exposing an overridable cipher_callback hook that returns success-without-effect is a footgun, especially for a crypto library where a silent no-op cipher is dangerous.
Fix: Until the cipher struct is actually exposed, either remove the cipher_callback hook or always return CRYPTOCB_UNAVAILABLE for cipher so wolfCrypt falls back to software, rather than returning 0/success for an operation that did nothing.
There was a problem hiding this comment.
Removed the cipher_callback definition. Can only be added back when the cffi code generation for the wc_CryptoInfo structure is made to work.
| from wolfcrypt.cryptocb import CryptoCallback | ||
|
|
||
|
|
||
| def test_default_device_id(): |
There was a problem hiding this comment.
🟠 [Medium] Hash callbacks and error paths are untested; default-device-id test asserts nothing
Only the RNG callback path is exercised. There is no coverage for hash_update_callback / hash_finalize_callback (the most complex branch in callback(), including the NULL-digest update vs finalize distinction and the digest-length validation), no coverage for the length-mismatch error branches (which is exactly where the success-on-exception bug above manifests), and no coverage for the cipher / unknown-algo fallback to CRYPTOCB_UNAVAILABLE. test_default_device_id only print()s the value and asserts nothing, so it cannot fail meaningfully.
Fix: Add tests for the hash update/finalize paths, the length-mismatch/error paths, and the unknown-algo fallback; make test_default_device_id assert on the return value.
There was a problem hiding this comment.
Changed the test_default_device_id so that it checks that the expected value (according to the configuration used for the Python version).
Added test to check support for crypto callbacks for hash functions. For this some infrastructure had to be added to the _Hash class to support device_ids needed to call the callbacks.
| self._unregister() | ||
| return False | ||
|
|
||
| def __del__(self) -> None: |
There was a problem hiding this comment.
🟠 [Medium] del unregister not guarded against interpreter-shutdown errors
__del__ calls self._unregister() which dereferences module-level _lib. During interpreter shutdown _lib/wc_CryptoCb_UnRegisterDevice may already be torn down, raising AttributeError (or worse) from a destructor, producing noisy 'Exception ignored in del' output. The existing Random.__del__ in wolfcrypt/random.py explicitly guards this exact scenario with a try/except for shutdown. The new code should follow the same established pattern. (Also note __exit__ followed by __del__ calls _unregister twice; this is harmless since unregister is idempotent, but the shutdown guard is still warranted.)
Fix: Guard the destructor's unregister against shutdown-time teardown, matching the pattern already used in Random.__del__.
There was a problem hiding this comment.
Please explain how this is an issue. It is not clear to me how the C-extension gets unloaded before this module.
I am lacking in knowledge/experience about this specific failure mode.
Is this explained somewhere in CFFI documentation?
| self._unregister() | ||
|
|
||
| def callback(self, device_id: int, info: _ffi.CData) -> int: | ||
| log.debug(f"{device_id=} algo = {ALGO_TYPE_NAME[info.algo_type]}") |
There was a problem hiding this comment.
🔵 [Low] Inconsistent logging style in callback (f-string vs lazy %-formatting)
Line 104 uses an eagerly-interpolated f-string inside log.debug(...), while line 109 uses lazy %s formatting. Mixing the two is inconsistent and the f-string form is what pylint's logging-fstring-interpolation flags (the file already disables several pylint checks at the top). Pick one style; lazy %-args avoid formatting cost when debug logging is disabled.
Fix: Use lazy %-style logging consistently across the callback.
| from wolfcrypt.cryptocb import CryptoCallback | ||
| from wolfcrypt.exceptions import WolfCryptApiError | ||
|
|
||
| ret = _lib.wolfCrypt_Init() |
There was a problem hiding this comment.
🔵 [Low] wolfCrypt_Init() added with no corresponding cleanup
wolfCrypt_Init() is now called unconditionally on package import (a reasonable change, and required for the cryptocb registration to work), but there is no matching wolfCrypt_Cleanup() registered (e.g. via atexit). For a process-lifetime library this is generally acceptable, but it is worth a deliberate decision/comment so it is clear cleanup was intentionally omitted rather than forgotten.
Fix: Confirm the omission of wolfCrypt_Cleanup() is intentional; add a brief comment or an atexit cleanup if appropriate.
There was a problem hiding this comment.
Added a comment that it is not needed to call this function. This function initializes a global data structure that lives as long as the application and gets cleaned up together with the process.
1f974ad to
1ab3833
Compare
roberthdevries
left a comment
There was a problem hiding this comment.
Not all issues are addressed. Will continue working on this after typing is merged so that this part can be typed as well.
| self._unregister() | ||
|
|
||
| def callback(self, device_id: int, info: _ffi.CData) -> int: | ||
| log.debug(f"{device_id=} algo = {ALGO_TYPE_NAME[info.algo_type]}") |
|
@roberthdevries now that the other PR's have been merged please take another look at the remaining items. Thanks! |
* Comment out code in build_ffi.py instead of quoted string. * Check return value of wolfCrypt_init() * Complete algorithm type hash table * Change dicts into defaultdicts with a sensible value when key is missing. * Convert debug print to debug log messages * Add length checks to produce nicer exceptions.
* Add if _lib.CRYPT_CB_ENABLED guards around code that depends on that configuration option being enabled.
* added various tests * adapted hash interface to allow callbacks
1ab3833 to
d2debf4
Compare
dgarske
left a comment
There was a problem hiding this comment.
Skoll Code Review
Scan type: reviewOverall recommendation: REQUEST_CHANGES
Findings: 8 total — 8 posted, 0 skipped
8 finding(s) posted as inline comments (see file-level comments below)
Posted findings
- [High] Import-time reference to _lib.INVALID_DEVID reintroduces issue #2659 regression —
wolfcrypt/hashes.py:43 - [High] device_id silently ignored for Sha256/Sha384/Sha512 (callback never fires) —
wolfcrypt/hashes.py:235-298 - [High] Uncaught callback exceptions report success (0) to wolfCrypt with an unfilled buffer —
wolfcrypt/__init__.py:64-69 - [Medium] Sha3 constructor does not expose device_id despite _init supporting it —
wolfcrypt/hashes.py:352-358 - [Medium] CryptoCallback.del not guarded for interpreter shutdown; double-unregister and self-reference cycle —
wolfcrypt/cryptocb.py:96-103 - [Medium] Missing test coverage for callback error paths and SHA-2 device_id —
tests/test_cryptocb.py:1-87 - [Low] Hash update-vs-finalize discriminator drops input when both in and digest are set —
wolfcrypt/cryptocb.py:112-126 - [Low] lib.pyi wc_HashInfo union field name does not match C member —
wolfcrypt/_ffi/lib.pyi:592-601
Review generated by Skoll
| **Hash Function Interface**. | ||
| """ | ||
| def __init__(self, string: BytesOrStr | None = None) -> None: | ||
| def __init__(self, string: BytesOrStr | None = None, device_id: int = _lib.INVALID_DEVID) -> None: |
There was a problem hiding this comment.
🔴 [High] Import-time reference to _lib.INVALID_DEVID reintroduces issue #2659 regression
The new default argument device_id: int = _lib.INVALID_DEVID is evaluated at module-import time (default parameter values are evaluated when the def in the _Hash class body executes). INVALID_DEVID is only emitted into the CFFI cdef when ML_KEM OR ML_DSA OR (newly) CRYPTO_CB is enabled (scripts/build_ffi.py:1317). In a wolfSSL build with none of those three (exactly the configuration the build-no-pqc CI job exercises: --enable-cryptonly ... --disable-mlkem --disable-mldsa and no --enable-cryptocb), _lib.INVALID_DEVID does not exist and import wolfcrypt.hashes raises AttributeError. This is the precise regression that issue #2659 fixed for wolfcrypt/random.py (which deliberately hardcodes device_id: int = -2 instead of _lib.INVALID_DEVID). This PR re-introduces the exact failure in hashes.py, breaking hashing for every no-PQC/no-cryptocb build and the build-no-pqc regression job. Line 358 (self._init(_lib.INVALID_DEVID)) has the same dependency but only fails at Sha3 instantiation time; line 43 fails at import, which is worse.
Fix: Follow the established random.py pattern: use the literal -2 for the default (and at hashes.py:358), or unconditionally declare INVALID_DEVID in the cdef so it is always available. Verify the build-no-pqc CI job passes.
There was a problem hiding this comment.
The second option is chosen. This removes the need for the magic value -2.
|
|
||
| @override | ||
| def _init(self) -> int: | ||
| def _init(self, device_id: int) -> int: |
There was a problem hiding this comment.
🔴 [High] device_id silently ignored for Sha256/Sha384/Sha512 (callback never fires)
_Hash.__init__ now threads device_id into _init(device_id), and the Sha class was updated to honor it via wc_InitSha_ex(obj, NULL, device_id). However Sha256._init, Sha384._init, and Sha512._init still call the non-_ex initializers wc_InitSha256(obj) / wc_InitSha384(obj) / wc_InitSha512(obj), which take no devId and default to INVALID_DEVID. As a result, Sha256(device_id=11) (and 384/512) accept the parameter but silently drop it, so the registered crypto callback is never invoked for those hash types. cryptocb.py advertises support for these types (they are present in DIGEST_SIZE/HASH_TYPE_NAME), so a user reasonably expects the callback to fire. Silently ignoring a security-relevant device-routing parameter is a correctness trap: the user believes offload/interception is active when it is not.
Fix: Add the _ex initializers to the cdef (wc_InitSha256_ex, wc_InitSha384_ex, wc_InitSha512_ex) and use them so device_id is honored. If SHA-2 device support is intentionally out of scope for this 'initial' PR, reject device_id != INVALID_DEVID for those classes (or raise) rather than silently ignoring it, and document the limitation.
There was a problem hiding this comment.
Used the _ex variants of the hash init functions and passed the device_id to them.
| raise WolfCryptApiError("WolfCrypt_Init failed", ret) | ||
|
|
||
| if _lib.CRYPTO_CB_ENABLED: | ||
| @_ffi.def_extern() |
There was a problem hiding this comment.
🔴 [High] Uncaught callback exceptions report success (0) to wolfCrypt with an unfilled buffer
CryptoCallback.callback only catches NotImplementedError; any other exception (e.g. the ValueError it deliberately raises when len(out) != info.rng.sz or the digest length is wrong) propagates out through the @_ffi.def_extern() function py_wc_crypto_callback. cffi's def_extern with no error=/onerror= argument prints the traceback to stderr and returns the default value 0 to the C caller. In the wolfCrypt crypto-callback convention 0 means 'handled successfully'. So a buggy user callback that returns the wrong number of bytes causes wolfCrypt to treat the operation as successful while the output buffer (info.rng.out / info.hash.digest) was never written. For RNG this yields uninitialized stack/heap memory being consumed as 'random' data. The length-check raise ValueError(...) is therefore not just ineffective, it is actively harmful because the error is swallowed into a success code.
Fix: Pass a negative error= (and/or onerror=) to @_ffi.def_extern() so an uncaught exception returns a hard failure to wolfCrypt, and/or wrap the body of callback in a broad except Exception: that logs and returns a negative error code instead of letting exceptions escape. Add a test that a wrong-length RNG/digest return does not report success.
There was a problem hiding this comment.
Added an onerror callback that checks the exception type for a WolfCryptApiError and then returns the error code of that exception, in all other cases it returns a generic -1 error code.
It will log an error message for debugging purposes with the original exception string.
Also added a test to check the correct handling of a ValueError.
| self._delete = self._SHA3_FREE.get(size) | ||
| self._copy = self._SHA3_COPY.get(size) | ||
| ret = self._init() | ||
| ret = self._init(_lib.INVALID_DEVID) |
There was a problem hiding this comment.
🟠 [Medium] Sha3 constructor does not expose device_id despite _init supporting it
Sha3._init was updated to forward device_id to wc_InitSha3_*(obj, NULL, device_id), but Sha3.__init__ has signature (self, string=None, size=...) with no device_id parameter and calls self._init(_lib.INVALID_DEVID) with a hardcoded value. So Sha3 can never actually route to a crypto callback, and its _init change is dead code from the public API's perspective. This is inconsistent with Sha (which does expose device_id) and with the DIGEST_SIZE table in cryptocb.py that lists SHA3 types.
Fix: Add a device_id parameter to Sha3.__init__ (and Sha3.new) and pass it to _init, or explicitly document that Sha3 device support is not yet wired. Also note line 358's _lib.INVALID_DEVID shares the import/eval-time dependency called out above.
There was a problem hiding this comment.
Added deviced_id parameter to Sha3.new and Sha3.__init__
| def __enter__(self) -> Self: | ||
| return self | ||
|
|
||
| def __exit__( |
There was a problem hiding this comment.
🟠 [Medium] CryptoCallback.del not guarded for interpreter shutdown; double-unregister and self-reference cycle
Three related cleanup concerns: (1) __del__ calls _unregister() -> _lib.wc_CryptoCb_UnRegisterDevice(...) with no protection against interpreter-shutdown teardown, whereas random.py:__del__ deliberately wraps its native free in try/except AttributeError because _lib may be partially torn down at shutdown; this can emit noisy exceptions during finalization. (2) When used as a context manager, __exit__ calls _unregister() and then __del__ calls it again -> the device is unregistered twice (harmless in wolfSSL but indicates missing idempotency/guard). (3) self.ctx = _ffi.new_handle(self) plus new_handle keeping self alive forms a reference cycle (self -> ctx -> self); combined with a user-defined __del__, objects created without with are only reclaimed by the cyclic GC, so the device stays registered longer than the caller expects.
Fix: Guard __del__ against shutdown like random.py, make _unregister idempotent (track a _registered flag), and consider clearing self.ctx on unregister to break the cycle for prompt cleanup.
| @@ -0,0 +1,87 @@ | |||
| # test_cryptocb.py | |||
There was a problem hiding this comment.
🟠 [Medium] Missing test coverage for callback error paths and SHA-2 device_id
The new tests cover only the happy path for RNG (SHA-1 hash) and default_device_id. There is no coverage for: (a) a callback returning the wrong number of bytes (the ValueError guard in callback - important given the def_extern success-on-exception issue above); (b) an unsupported algo/hash type returning CRYPTOCB_UNAVAILABLE; (c) Sha256/Sha384/Sha512 with device_id set (which would have surfaced that device_id is silently ignored for those classes); (d) the not-implemented default methods raising NotImplementedError. These gaps let the two SHA-2 and exception-handling bugs above pass CI.
Fix: Extend test_cryptocb.py to cover error paths, unsupported-algo fallthrough, and SHA-256/384/512 device routing.
| if info.hash.type not in DIGEST_SIZE: | ||
| return _lib.CRYPTOCB_UNAVAILABLE | ||
| log.debug("hash_type = %s", HASH_TYPE_NAME[info.hash.type]) | ||
| if info.hash.digest == _ffi.NULL: |
There was a problem hiding this comment.
🔵 [Low] Hash update-vs-finalize discriminator drops input when both in and digest are set
The callback distinguishes update from finalize purely via info.hash.digest == _ffi.NULL. wolfCrypt's one-shot hash paths (e.g. wc_ShaHash) can invoke the crypto-cb hash entry with BOTH a non-NULL input (data/data_size) and a non-NULL digest in a single call. In that case this code takes the else (finalize) branch and never delivers the input data to hash_update_callback, producing a digest over missing data. The wolfcrypt-py _Hash object happens to call update and final separately so the tested path is safe, but a one-shot caller would be mis-hashed.
Fix: Handle update and finalize independently within a single invocation (process input if present, then finalize if digest present), or confirm wolfCrypt never passes both for the supported hash types.
| def wc_MlDsaKey_GetPubLen(key: DilithiumKey, len: IntPtr) -> int: ... | ||
| def wc_MlDsaKey_GetSigLen(key: DilithiumKey, len: IntPtr) -> int: ... | ||
|
|
||
| @dataclass |
There was a problem hiding this comment.
🔵 [Low] lib.pyi wc_HashInfo union field name does not match C member
The type stub declares the SHA-1 union member as sha: FFI.CData, but the cdef in build_ffi.py names it sha1 (wc_Sha* sha1;). The runtime code never accesses these union members so there is no functional impact, but the stub is inconsistent with the actual struct and would mislead anyone typing against info.hash.sha1.
Fix: Rename the stub field to sha1 to match the cdef, or align the cdef member name with the stub.
This makes it also no longer necessary to use the hard-coded magic value -2 as default argument for various cases where it was already required.
|
Now that the |
roberthdevries
left a comment
There was a problem hiding this comment.
Not all review comments addressed. More time needed.
| self._delete = self._SHA3_FREE.get(size) | ||
| self._copy = self._SHA3_COPY.get(size) | ||
| ret = self._init() | ||
| ret = self._init(_lib.INVALID_DEVID) |
There was a problem hiding this comment.
Added deviced_id parameter to Sha3.new and Sha3.__init__
|
|
||
| @override | ||
| def _init(self) -> int: | ||
| def _init(self, device_id: int) -> int: |
There was a problem hiding this comment.
Used the _ex variants of the hash init functions and passed the device_id to them.
| raise WolfCryptApiError("WolfCrypt_Init failed", ret) | ||
|
|
||
| if _lib.CRYPTO_CB_ENABLED: | ||
| @_ffi.def_extern() |
There was a problem hiding this comment.
Added an onerror callback that checks the exception type for a WolfCryptApiError and then returns the error code of that exception, in all other cases it returns a generic -1 error code.
It will log an error message for debugging purposes with the original exception string.
Also added a test to check the correct handling of a ValueError.
| def wc_MlDsaKey_GetPubLen(key: DilithiumKey, len: IntPtr) -> int: ... | ||
| def wc_MlDsaKey_GetSigLen(key: DilithiumKey, len: IntPtr) -> int: ... | ||
|
|
||
| @dataclass |
This patch does not add support for types of callbacks. Adding support for callbacks for cipher functions is complicated by issues with cffi code generation for the
wc_CryptoInfostructure with two layers of anonymous unions.Uncommenting more parts of the cipher struct causes errors regarding conflicting struct sizes of other parts of the
wc_CryptoInfostruct.Cffi
cdefand the compiler seem to disagree.However as it is, callbacks work for random and hash functions.