-
Notifications
You must be signed in to change notification settings - Fork 122
feat: Adds thread safe Settings and Context support to c_ffi_api #1783
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
…and update stream testing code.
New C FFI Functions Added: c2pa_settings_new() - Creates a new settings object c2pa_settings_update_from_string() - Updates settings from JSON/TOML string c2pa_settings_set_value() - Sets individual config values using dot notation (with JSON parsing) c2pa_context_new() - Creates a new context object c2pa_context_set_settings() - Applies settings to a context c2pa_free() - Universal free function for all C2PA objects There are also new set apis in Rust corresponding to Contex-with apis. This is requirement for bindings
Merging this PR will not alter performance
Comparing Footnotes
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1783 +/- ##
==========================================
- Coverage 75.66% 75.48% -0.19%
==========================================
Files 168 170 +2
Lines 38893 39242 +349
==========================================
+ Hits 29428 29620 +192
- Misses 9465 9622 +157 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
| #[no_mangle] | ||
| pub unsafe extern "C" fn c2pa_error() -> *mut c_char { | ||
| to_c_string(Error::last_message().unwrap_or_default()) | ||
| to_c_string(Error::last_message()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No default anymore?
c2pa_c_ffi/src/c_api.rs
Outdated
| // TODO: Reader::from_context takes ownership of Context, but Context doesn't implement Clone | ||
| // and we can't take ownership from a *const pointer. Need to decide on the correct approach: | ||
| // 1. Change signature to *mut and actually take ownership from C | ||
| // 2. Use from_shared_context instead |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Isn't that what would make the most sense? Since I assume context may be passed around potentially in this layer?
| #[derive(Error, Debug)] | ||
| /// Defines all possible errors that can occur in this library | ||
| pub enum Error { | ||
| pub enum C2paError { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Type name change for namespacing?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pretty much, neither of these are directly exported, but this helps keep us from clashing with other Error instances.
| /// | ||
| /// 0 on success, negative value on error. | ||
| #[no_mangle] | ||
| pub unsafe extern "C" fn c2pa_settings_set_value( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That is nice, but agreed a settings builder pattern would be nicer! (Albeit the params & ref handling can be tricky, as I am discovering too).
…le making later access immutable. New C FFI Functions (in c2pa_c_ffi/src/c_api.rs): Context Builder Pattern: c2pa_context_builder_new() - Creates a new mutable context builder c2pa_context_builder_set_settings() - Configures the builder with settings c2pa_context_builder_build() - Builds an immutable Arc<Context> from the builder c2pa_context_new() - Convenience function for creating a default immutable context Reader Functions: c2pa_reader_from_context() - Creates a Reader from a shared context (context can be reused) c2pa_reader_with_stream() - Configures an existing reader with a stream (consumes the reader) Supporting Infrastructure: c2pa_free() - Universal free function for all tracked pointers (was already added earlier) c2pa_settings_new() - Creates a new settings object c2pa_settings_update_from_string() - Updates settings from JSON/TOML c2pa_settings_set_value() - Sets individual config values using dot notation Type Aliases: C2paContext = Arc<Context> - Immutable, shareable context C2paContextBuilder = Context - Mutable builder for construction
| /// Any thumbnails or other binary resources will be written to data_dir if provided. | ||
| /// # Safety | ||
| /// | ||
| /// This function is safe to call. The returned pointer must be freed |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| /// This function is safe to call. The returned pointer must be freed | |
| /// The returned pointer must be freed |
|
|
||
| // Internal routine to convert a *const c_char to a rust String or return a -1 int error. | ||
| #[macro_export] | ||
| macro_rules! cstr_or_return_int { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same as cstr_or_return_neg?
scouten-adobe
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider this a "request changes" status, but not using that status in case I'm not available to re-review.
|
|
||
| // Unused - kept for potential future use | ||
| #[allow(unused_macros)] | ||
| macro_rules! some_or_return_other_null { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Naming request: some_or_return_other makes sense when you're providing the "other" value.
For macros such as this where the macro name specifies the None case, can I suggest naming as some_or_return_null (omitting the other part)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Applies to several macros in this file.
| /// Track a pointer with its type and cleanup function | ||
| fn track(&self, ptr: usize, type_id: TypeId, cleanup: CleanupFn) { | ||
| if ptr != 0 { | ||
| self.tracked.lock().unwrap().insert(ptr, (type_id, cleanup)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add a comment here explaining why the unwrap() should be considered safe.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(Or add a panic handler? TO turn panics into errors for that FFI layer?)
| fn drop(&mut self) { | ||
| let tracked = self.tracked.lock().unwrap(); | ||
| if !tracked.is_empty() { | ||
| eprintln!( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Very nice!
tmathern
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(Please disregard thread-safety related comment if thread safety is not a topic here).
| // Caller must ensure ptr is valid for reading and points to a | ||
| // null-terminated string within MAX_CSTRING_LEN bytes. | ||
| let bytes = unsafe { | ||
| std::slice::from_raw_parts(ptr as *const u8, $crate::macros::MAX_CSTRING_LEN) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Isn't that always exactly the size MAX_CSTRING_LEN, even if our string is less, like if the expected size let's say is MAX_CSTRING_LEN/2? Then aren't we over-allocating? Is it fine or undefined behavior then?
| self.validate(ptr as usize, TypeId::of::<T>())?; | ||
|
|
||
| // Temporarily swap out the value (no clone needed!) | ||
| let value = std::ptr::replace(ptr, T::default()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we want thread safety? We may need a guard/lock here so another thread couldn't be in this code with a pointer in parallel
| let mutex = &*ptr; | ||
|
|
||
| // Lock and swap - all atomic under the lock | ||
| let mut guard = mutex.lock().unwrap(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Poisoned mutex would panic, should we return an error instead?
| /// ```ignore | ||
| /// let ptr = track_box(Box::into_raw(Box::new(value))); | ||
| /// ``` | ||
| pub fn track_box<T: 'static>(ptr: *mut T) -> *mut T { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| pub fn track_box<T: 'static>(ptr: *mut T) -> *mut T { | |
| pub fn track_box<T: 'static + Send>(ptr: *mut T) -> *mut T { |
Do we want thread safety?
This captures the usize, which I think is Send. If T is !Send, because the closure using usize is Send, we could end up still running it. So we could end up freeing a !Send on another thread. Are we even worried about this case?
| /// This function performs pointer arithmetic with `ptr.add(size)` which requires | ||
| /// that the pointer and size are valid for the memory region being checked. | ||
| pub unsafe fn is_safe_buffer_size(size: usize, ptr: *const c_uchar) -> bool { | ||
| // Combined checks for early return - improves branch prediction |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| // Combined checks for early return - improves branch prediction | |
| // Combined checks for early return - improves branch prediction | |
| // 0-length buffer is considered unsafe, since used by C FFI layer |
| /// The returned pointer must be freed exactly once by calling `free_c_bytes` | ||
| pub fn to_c_bytes(bytes: Vec<u8>) -> *const c_uchar { | ||
| let len = bytes.len(); | ||
| let ptr = Box::into_raw(bytes.into_boxed_slice()) as *const c_uchar; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In theory, bytes could be 0 sized. Since we deal with the C level here, this may create dangling pointers then (0-sized allocation). In another place we said o-sized things are not allowed, we should stay consistent.
| /// Track a pointer with its type and cleanup function | ||
| fn track(&self, ptr: usize, type_id: TypeId, cleanup: CleanupFn) { | ||
| if ptr != 0 { | ||
| self.tracked.lock().unwrap().insert(ptr, (type_id, cleanup)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(Or add a panic handler? TO turn panics into errors for that FFI layer?)
tmathern
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(Can we leak in errors paths, since we expect caller error handling/invalidation? That is a recurring question for FFI layers... how to handle the memory ine rror paths, since it has technically all become invalid... due to the error...).
| check_or_return_int!(dest); | ||
| check_or_return_int!(signer_ptr); | ||
| check_or_return_int!(manifest_bytes_ptr); | ||
| ptr_or_return_int!(builder_ptr); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need to check if these dereferenced pointers are some we track, or we allow any?
| tsa_url: *const c_char, | ||
| ) -> *mut C2paSigner { | ||
| let certs = from_cstr_or_return_null!(certs); | ||
| let certs = cstr_or_return_null!(certs); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Isn't tracking missing here somewhere? I'd expect tracking starts here for a signer pointer, possibly?
| ) -> *mut C2paBuilder { | ||
| // Take ownership of the builder (consumes it) | ||
| let builder = Box::from_raw(builder); | ||
| let manifest_json = cstr_or_return_null!(manifest_json); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think if we return/error early before the box_tracked call, we leak the builder pointer
New C2paSettings and C2paContext along with related Reader and Builder updates.