Thread Safety Analysis of Shared Memory APIs

[owny990312]

Describe the feature

Describe the feature
The question concerns the thread safety of shared memory interface APIs, specifically:

z_shm_provider_garbage_collect
z_shm_provider_defragment
z_shm_provider_alloc
z_publisher_put
In a multi-threaded environment where multiple publishers with the same topic are publishing data, are these APIs thread safe?

[yellowhatter]

Hello, @owny990312 !
This is a tricky question.
The thread safety of shm provider and allocation layout API's depend on provider's backend. If the backend is thread-safe, the whole provider with all it's API will also be thread-safe.
Our general-purpose PosixShmProviderBackend is thread-safe.
If you are using your own C-callback-based backend implementation, the thread-safety will depend on it

SHM Api is marked unstable and thus we are still polishing it. Your question raises a good point to improve, thank you

[owny990312]

Hello, @owny990312 ! This is a tricky question.这是一个棘手的问题。 The thread safety of shm provider and allocation layout API's depend on provider's backend. If the backend is thread-safe, the whole provider with all it's API will also be thread-safe.shm 提供者和分配布局 API 的线程安全性取决于提供者的后端。如果后端是线程安全的，那么整个提供者及其所有 API 也将是线程安全的。 Our general-purpose PosixShmProviderBackend is thread-safe.我们的通用 PosixShmProviderBackend 是线程安全的。 If you are using your own C-callback-based backend implementation, the thread-safety will depend on it如果你使用的是自己的基于 C 回调的后端实现，线程安全性将取决于它

SHM Api is marked unstable and thus we are still polishing it. Your question raises a good point to improve, thank youSHM Api 标记为不稳定，因此我们仍在对其进行完善。您的问题提出了一个很好的改进点，谢谢。

Thanls for your reply.

How to set its provider's backend, if we use

    z_posix_shm_provider_new(&shm_provider_, z_loan(shm_layout_));

it means we set general-purpose PosixShmProviderBackend ?

[yellowhatter]

Hello, @owny990312 ! This is a tricky question.这是一个棘手的问题。 The thread safety of shm provider and allocation layout API's depend on provider's backend. If the backend is thread-safe, the whole provider with all it's API will also be thread-safe.shm 提供者和分配布局 API 的线程安全性取决于提供者的后端。如果后端是线程安全的，那么整个提供者及其所有 API 也将是线程安全的。 Our general-purpose PosixShmProviderBackend is thread-safe.我们的通用 PosixShmProviderBackend 是线程安全的。 If you are using your own C-callback-based backend implementation, the thread-safety will depend on it如果你使用的是自己的基于 C 回调的后端实现，线程安全性将取决于它
SHM Api is marked unstable and thus we are still polishing it. Your question raises a good point to improve, thank youSHM Api 标记为不稳定，因此我们仍在对其进行完善。您的问题提出了一个很好的改进点，谢谢。

Thanls for your reply.

How to set its provider's backend, if we use

    z_posix_shm_provider_new(&shm_provider_, z_loan(shm_layout_));

it means we set general-purpose PosixShmProviderBackend ?

Yes, here you are using our general-purpose POSIX backed. And It is guaranteed (now and in the future) to be everything-safe

[owny990312]

Hello, @owny990312 ! This is a tricky question.这是一个棘手的问题。 The thread safety of shm provider and allocation layout API's depend on provider's backend. If the backend is thread-safe, the whole provider with all it's API will also be thread-safe.shm 提供者和分配布局 API 的线程安全性取决于提供者的后端。Hello, @owny990312 ! 这是一个棘手的问题。shm 提供者和分配布局 API 的线程安全性取决于提供者的后端。如果后端是线程安全的，那么整个提供者及其所有 API 也将是线程安全的。如果后端是线程安全的，那么整个提供者及其所有 API 也将是线程安全的。 Our general-purpose PosixShmProviderBackend is thread-safe.我们的通用 PosixShmProviderBackend 是线程安全的。 If you are using your own C-callback-based backend implementation, the thread-safety will depend on it如果你使用的是自己的基于 C 回调的后端实现，线程安全性将取决于它
SHM Api is marked unstable and thus we are still polishing it. Your question raises a good point to improve, thank youSHM Api 标记为不稳定，因此我们仍在对其进行完善。您的问题提出了一个很好的改进点，谢谢。SHM Api 标记为不稳定，因此我们仍在对其进行完善。您的问题提出了一个很好的改进点，谢谢。

Thanls for your reply.  谢谢您的回复。
How to set its provider's backend, if we use如何设置其提供者的后端，如果我们使用

    z_posix_shm_provider_new(&shm_provider_, z_loan(shm_layout_));

it means we set general-purpose PosixShmProviderBackend ?它意味着我们设置了通用的 PosixShmProviderBackend？

Yes, here you are using our general-purpose POSIX backed. And It is guaranteed (now and in the future) to be everything-safe是的，您正在使用我们的一般用途的 POSIX 兼容版本。并且现在和将来都保证是安全的。

Thanks for your reply

[yellowhatter]

@owny990312 However, if your system design might produce high contention around the provider (a lot of overlapping concurrent calls of it's API), maybe it is worth thinking about having a provider's instance per thread, otherwise the contention might degrade the performance. We have plans to optimize our default provider, and these plans also include good improvements for described scenario, but currently it is not perfect

[owny990312]

@owny990312 However, if your system design might produce high contention around the provider (a lot of overlapping concurrent calls of it's API), maybe it is worth thinking about having a provider's instance per thread, otherwise the contention might degrade the performance.然而，如果你的系统设计可能会在提供者周围产生高竞争（大量的重叠并发调用其 API），那么考虑为每个线程拥有一个提供者的实例可能是值得的，否则竞争可能会降低性能。 We have plans to optimize our default provider, and these plans also include good improvements for described scenario, but currently it is not perfect我们计划优化默认提供商，并且这些计划也包括对所述场景的良好改进，但目前还不完美

Thanks for your reply. And I want to know if the following API is also threrad-safe:

z_owned_bytes_t z_payload;
z_bytes_from_buf(&z_payload, static_cast<uint8_t *>(ptr), size, nullptr, nullptr);
z_publisher_put(z_loan(zenoh_pub_), z_move(z_payload), &zenoh_pub_opts_);

[DenisBiryukov91]

Thanks for your reply. And I want to know if the following API is also threrad-safe:

z_owned_bytes_t z_payload; z_bytes_from_buf(&z_payload, static_cast<uint8_t *>(ptr), size, nullptr, nullptr); z_publisher_put(z_loan(zenoh_pub_), z_move(z_payload), &zenoh_pub_opts_);

Yes z_publisher_put is thread safe (as generally all operations on zenoh entities that use const pointers to them). You should never try to modify the data pointed by ptr though, once you passed it to z_bytes_from_buf this way, even after z_publisher_put returns (you can pass a non-null deleter though which can signal when the data is no longer in use by zenoh).

[owny990312]

Thanks for your reply. And I want to know if the following API is also threrad-safe:谢谢您的回复。我想知道以下 API 是否也是线程安全的：
z_owned_bytes_t z_payload; z_bytes_from_buf(&z_payload, static_cast<uint8_t *>(ptr), size, nullptr, nullptr); z_publisher_put(z_loan(zenoh_pub_), z_move(z_payload), &zenoh_pub_opts_);z_owned_bytes_t z_payload; z_bytes_from_buf(&z_payload, static_cast(ptr), size, nullptr, nullptr); z_publisher_put(z_loan(zenoh_pub_), z_move(z_payload), &zenoh_pub_opts_);

Yes z_publisher_put is thread safe (as generally all operations on zenoh entities that use const pointers to them). You should never try to modify the data pointed by ptr though, once you passed it to z_bytes_from_buf this way.z_publisher_put 是线程安全的（正如所有使用 const 指针操作 zenoh 实体的操作一般都线程安全一样）。不过，你永远不应该尝试修改 ptr 指向的数据，因为你通过这种方式将其传递给 z_bytes_from_buf 之后。

Do you mean that the parameters indicated by the arrows in the figure—those accessed via const-qualified sample pointers—are almost always thread-safe?

[DenisBiryukov91]

Thanks for your reply. And I want to know if the following API is also threrad-safe:谢谢您的回复。我想知道以下 API 是否也是线程安全的：
z_owned_bytes_t z_payload; z_bytes_from_buf(&z_payload, static_cast<uint8_t *>(ptr), size, nullptr, nullptr); z_publisher_put(z_loan(zenoh_pub_), z_move(z_payload), &zenoh_pub_opts_);z_owned_bytes_t z_payload; z_bytes_from_buf(&z_payload, static_cast(ptr), size, nullptr, nullptr); z_publisher_put(z_loan(zenoh_pub_), z_move(z_payload), &zenoh_pub_opts_);

Yes z_publisher_put is thread safe (as generally all operations on zenoh entities that use const pointers to them). You should never try to modify the data pointed by ptr though, once you passed it to z_bytes_from_buf this way.z_publisher_put 是线程安全的（正如所有使用 const 指针操作 zenoh 实体的操作一般都线程安全一样）。不过，你永远不应该尝试修改 ptr 指向的数据，因为你通过这种方式将其传递给 z_bytes_from_buf 之后。

Do you mean that the parameters indicated by the arrows in the figure—those accessed via const-qualified sample pointers—are almost always thread-safe?

yes, this is how it is supposed to be.

[owny990312]

Thanks for your reply. And I want to know if the following API is also threrad-safe:谢谢您的回复。我想知道以下 API 是否也是线程安全的：谢谢您的回复。我想知道以下 API 是否也是线程安全的：
z_owned_bytes_t z_payload; z_bytes_from_buf(&z_payload, static_cast<uint8_t *>(ptr), size, nullptr, nullptr); z_publisher_put(z_loan(zenoh_pub_), z_move(z_payload), &zenoh_pub_opts_);z_owned_bytes_t z_payload; z_bytes_from_buf(&z_payload, static_cast(ptr), size, nullptr, nullptr); z_publisher_put(z_loan(zenoh_pub_), z_move(z_payload), &zenoh_pub_opts_);z_owned_bytes_t z_payload; z_bytes_from_buf(&z_payload, static_cast(ptr), size, nullptr, nullptr); z_publisher_put(z_loan(zenoh_pub_), z_move(z_payload), &zenoh_pub_opts_);z_owned_bytes_t z_payload; z_bytes_from_buf(&z_payload, static_cast(ptr), size, nullptr, nullptr); z_publisher_put(z_loan(zenoh_pub_), z_move(z_payload), &zenoh_pub_opts_);

Yes z_publisher_put is thread safe (as generally all operations on zenoh entities that use const pointers to them). You should never try to modify the data pointed by ptr though, once you passed it to z_bytes_from_buf this way.z_publisher_put 是线程安全的（正如所有使用 const 指针操作 zenoh 实体的操作一般都线程安全一样）。不过，你永远不应该尝试修改 ptr 指向的数据，因为你通过这种方式将其传递给 z_bytes_from_buf 之后。z_publisher_put 是线程安全的（正如所有使用 const 指针操作 zenoh 实体的操作一般都线程安全一样）。不过，你永远不应该尝试修改 ptr 指向的数据，因为你通过这种方式将其传递给 z_bytes_from_buf 之后。

Do you mean that the parameters indicated by the arrows in the figure—those accessed via const-qualified sample pointers—are almost always thread-safe?您是说，图中箭头指示的参数——通过 const 标记的样本指针访问的那些参数——几乎总是线程安全的？

yes, this is how it is supposed to be.是的，这就是它应有的样子。

Thank you very much!