Slipstream Plugin Integration/Interface by dgrkotsonis · Pull Request #3190 · ProvableHQ/snarkVM

dgrkotsonis · 2026-03-16T13:42:17Z

Overview

Adds a Slipstream Plugin system for streaming real-time mappings and staking data to external services without relying on RPC calls.

Related PRs/Repos:

SnarkOS PR
Slipstream Postgres Plugin Example - Reach out to @dgrkotsonis to grant access

New Crates

`plugins/slipstream_plugin_interface`

Defines the SlipstreamPlugin trait — the interface all plugins must implement.

Method	Description
`on_load` / `on_unload`	Lifecycle hooks
`notify_mapping_update`	Called when a mapping key-value is inserted/updated during canonical finalize; args are serialized to bytes for object-safety
`notify_staking_reward`	Called once per staker per block during staking reward distribution
`history_enabled` / `history_staking_rewards_enabled`	Flags plugins use to opt in to data streams

Note: At the moment, any mappings filtering must be handled by the plugins themselves, notify_mapping_update gets called for all mappings

`plugins/slipstream_plugin_manager`

Manages loaded plugins and their backing libloading::Library handles.

LoadedSlipstreamPlugin — wrapper holding a boxed plugin + its name; implements Deref/DerefMut
SlipstreamPluginManager
- unload() — fires on_unload() on each plugin then drops the libraries
- history_mappings_enabled() / history_staking_rewards_enabled() — aggregate opt-in checks
- notify_mapping_update() — fan-out broadcast to all interested plugins
SlipstreamService — async service wrapping the manager (separate file)

Plugin Config File (JSON5)

Each plugin requires a config file:

{
  "libpath": "/path/to/libmy_plugin.so",  // required; relative paths resolve from the config file's dir
  "name": "my_plugin"                      // optional; overrides the plugin's name() return value
}

Plugin Library Convention

The shared library (.so / .dylib / .dll) must export a C function:

#[no_mangle]
pub extern "C" fn _create_plugin() -> *mut dyn SlipstreamPlugin {
    Box::into_raw(Box::new(MyPlugin::new()))
}

Startup

SlipstreamPluginService::new() takes a slice of config file paths:

let service = SlipstreamPluginService::new(&[
    PathBuf::from("/etc/aleo/plugins/my_plugin.json5"),
])?;

Note: Errors from plugin callbacks (notify_mapping_update, notify_staking_reward) are logged as warnings and never propagated — a misbehaving plugin will not crash the node.

Copilot

Pull request overview

Adds a Slipstream plugin system to stream canonical mapping updates and staking rewards to external services via dynamically loaded plugins, avoiding RPC-based polling.

Changes:

Introduces new plugin interface + plugin manager crates (dynamic loading via libloading, JSON5 config parsing).
Wires canonical-finalize lifecycle signaling into the VM + finalize store so storage writes can fan out events to plugins.
Extends ledger/store feature flags to include the plugin manager dependency for history and history-staking-rewards.

Reviewed changes

Copilot reviewed 13 out of 14 changed files in this pull request and generated 16 comments.

Show a summary per file

File	Description
utilities/src/bytes.rs	Minor refactor using `is_multiple_of(8)` for clarity.
synthesizer/src/vm/finalize.rs	Sets/reset canonical-finalize mode flag; emits staking reward notifications; some whitespace-only edits.
ledger/store/src/program/finalize.rs	Adds finalize-mode flag + plugin manager installation; notifies plugins on mapping updates and staking rewards.
ledger/store/Cargo.toml	`history*` features now pull in slipstream plugin manager.
plugins/slipstream_plugin_interface/src/slipstream_plugin_interface.rs	Defines the `SlipstreamPlugin` trait and callback surface.
plugins/slipstream_plugin_interface/src/lib.rs	Exposes the interface module.
plugins/slipstream_plugin_interface/README.md	Documents plugin config and library export convention.
plugins/slipstream_plugin_interface/Cargo.toml	New crate manifest for the interface.
plugins/slipstream_plugin_manager/src/slipstream_manager.rs	Implements plugin loading/unloading + fan-out notification logic + tests.
plugins/slipstream_plugin_manager/src/slipstream_service.rs	Adds a small service wrapper around the manager.
plugins/slipstream_plugin_manager/src/lib.rs	Module exports for manager/service.
plugins/slipstream_plugin_manager/Cargo.toml	New crate manifest for the manager.
Cargo.toml	Adds the new plugin crates to the workspace and workspace dependencies.
Cargo.lock	Adds transitive deps (e.g., `json5`, `pest*`) and the new crates.

plugins/slipstream_plugin_manager/src/slipstream_manager.rs

ledger/store/src/program/finalize.rs

synthesizer/src/vm/finalize.rs

plugins/slipstream_plugin_manager/src/slipstream_service.rs

plugins/slipstream_plugin_manager/src/slipstream_manager.rs

ledger/store/Cargo.toml

…y issue

vicsn

Mostly LGTM! Few questions

vicsn · 2026-04-04T13:43:11Z

ledger/store/src/program/finalize.rs

+        #[cfg(all(feature = "history", feature = "slipstream-plugins"))]
+        if let Some((pid, mname, k, v)) = plugin_data {
+            #[cfg(feature = "history")]
+            let height = self.storage.current_block_height().load(Ordering::SeqCst);


Is this used?

vicsn · 2026-04-04T13:45:27Z

ledger/store/src/program/finalize.rs

+        #[cfg(all(feature = "history", feature = "slipstream-plugins"))]
+        if let Some((pid, mname, serialized_entries)) = plugin_data {
+            #[cfg(feature = "history")]
+            let height = self.storage.current_block_height().load(Ordering::SeqCst);


Same note as above

vicsn · 2026-04-04T13:47:48Z

plugins/slipstream_plugin_interface/README.md

+])?;
+```
+
+> **Note:** Not yet wired up to any CLI flags or environment variables. How/where `SlipstreamPluginService` gets constructed and passed into the VM still needs to be plumbed in (likely in snarkOS or wherever the VM is instantiated).


Should we reword/remove?

vicsn · 2026-04-04T13:48:50Z

plugins/slipstream_plugin_interface/README.md

+
+> **Note:** Not yet wired up to any CLI flags or environment variables. How/where `SlipstreamPluginService` gets constructed and passed into the VM still needs to be plumbed in (likely in snarkOS or wherever the VM is instantiated).
+
+> Errors from plugin callbacks (`notify_mapping_update`, `notify_staking_reward`) are logged as warnings and never propagated — a misbehaving plugin will not crash the node.


I personally like the "fail loudly" philosophy. But proper error handling/metrics is also more work so can be left for the future.

vicsn · 2026-04-04T13:54:12Z

synthesizer/src/vm/finalize.rs

+        // Signal to Slipstream plugins that canonical finalize is starting.
+        #[cfg(feature = "slipstream-plugins")]
+        {
+            tracing::info!(target: "slipstream", "atomic_finalize: is_finalize_mode → true");


Is debug maybe a better log level?

Can/should this code block be hidden inside of the atomic_finalize! macro?

vicsn · 2026-04-04T13:57:37Z

plugins/slipstream_plugin_manager/src/slipstream_manager.rs

+// limitations under the License.
+
+use snarkvm_slipstream_plugin_interface::slipstream_plugin_interface::SlipstreamPlugin;
+use tokio::sync::oneshot::Sender as OneShotSender;


Nit: separate deps local to this repo and external to this repo with a newline

dgrkotsonis marked this pull request as draft March 16, 2026 13:42

dgrkotsonis force-pushed the stream_plugin_testing branch from 0341a32 to b77006b Compare March 16, 2026 14:00

dgrkotsonis changed the title ~~save place to check on first iteration of diffs~~ First Iteration of Stream Plugin (Draft) Mar 16, 2026

dgrkotsonis changed the title ~~First Iteration of Stream Plugin (Draft)~~ Slipstream Plugin Integration/Interface (Draft) Mar 16, 2026

dgrkotsonis mentioned this pull request Mar 17, 2026

[Feature/Draft] Add Slipstream Plugin Support ProvableHQ/snarkOS#4175

Draft

miazn requested a review from Copilot March 20, 2026 15:17

Copilot started reviewing on behalf of miazn March 20, 2026 15:18 View session

Copilot AI reviewed Mar 20, 2026

View reviewed changes

dgrkotsonis force-pushed the stream_plugin_testing branch from adf0c33 to 77faa25 Compare March 27, 2026 17:28

dgrkotsonis marked this pull request as ready for review April 1, 2026 21:21

dgrkotsonis changed the title ~~Slipstream Plugin Integration/Interface (Draft)~~ Slipstream Plugin Integration/Interface Apr 1, 2026

dgrkotsonis requested a review from ljedrz April 1, 2026 21:26

dgrkotsonis added 18 commits April 3, 2026 13:58

save place to check on first iteration of diffs

c3c2429

fix license, remove comments from prep

0b91215

remove comments, add type for serialized mappings to get rid of clipp…

027a403

…y issue

add more to readme

2820dfc

Add method to get slipstream plugin manager

c96eba5

add diagnostic log

9cd55f8

add new feature flag, debug logs

4cccb46

Adding log to staking notifs

2a2888e

add additional logs

68b234c

add more logging

f4af5c6

propagate tracing to interface

a7af4ce

More logs

ca43ff7

fix esc

5666186

wrap plugin manager in arc

cf0bed8

remove diagnostic logs

32dfed1

resolve PR review comments (first batch)

0c9105d

fix imports

802c996

fix another feature gate

17e347c

dgrkotsonis added 15 commits April 3, 2026 13:59

Resolve more PR comments

3cf0ffa

attempt to resolve error with loading duplicate plugin

a7b7cac

make slipstream logs debug, get rid of noise for now

939d34b

Update lockfile after rebase

fc27c57

recompiled

261abcb

remove noisy debug logs

bc5039f

Add diagnostic logs for dynamic loading/unloading

0d0aa19

temporarily remove dynamic reloading of the plugins

6dd2b78

update plugin version

38f5714

resolve some clippy issues

0246766

Cargo fmt

3ac009c

gate imports to resolve clippy error

74e2950

ran fmt in plugins

e22c364

formatting

bc2c61f

recommitted lockfile

933fbf9

dgrkotsonis force-pushed the stream_plugin_testing branch from 7fb6b76 to 933fbf9 Compare April 3, 2026 18:04

vicsn reviewed Apr 4, 2026

View reviewed changes


		> Note: Not yet wired up to any CLI flags or environment variables. How/where `SlipstreamPluginService` gets constructed and passed into the VM still needs to be plumbed in (likely in snarkOS or wherever the VM is instantiated).

		> Errors from plugin callbacks (`notify_mapping_update`, `notify_staking_reward`) are logged as warnings and never propagated — a misbehaving plugin will not crash the node. No newline at end of file

Conversation

dgrkotsonis commented Mar 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Related PRs/Repos:

New Crates

plugins/slipstream_plugin_interface

plugins/slipstream_plugin_manager

Plugin Config File (JSON5)

Plugin Library Convention

Startup

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vicsn left a comment

Choose a reason for hiding this comment

Uh oh!

vicsn Apr 4, 2026

Choose a reason for hiding this comment

Uh oh!

vicsn Apr 4, 2026

Choose a reason for hiding this comment

Uh oh!

vicsn Apr 4, 2026

Choose a reason for hiding this comment

Uh oh!

vicsn Apr 4, 2026

Choose a reason for hiding this comment

Uh oh!

vicsn Apr 4, 2026

Choose a reason for hiding this comment

Uh oh!

vicsn Apr 4, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

dgrkotsonis commented Mar 16, 2026 •

edited

Loading

`plugins/slipstream_plugin_interface`

`plugins/slipstream_plugin_manager`