-
Notifications
You must be signed in to change notification settings - Fork 17
feat(seatbelt): introduce chaos injection middleware #335
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 10 commits
Commits
Show all changes
16 commits
Select commit
Hold shift + click to select a range
1c6564e
feat(seatbelt): introduce chaos injection middleware
martintmk f0f3995
Update lib.rs
martintmk 47a513f
Update README.md
martintmk 6d5aaea
PR comments
martintmk 0990195
release new version
martintmk 292e392
Convert injection module tests to insta snapshots
martintmk 981cb58
ergonomic apis for errors
martintmk a8bf069
Update layer.rs
martintmk 1552672
Update README.md
martintmk 7812dd6
cleanup
martintmk 6b3f061
allow dynamically adjusting error rate
martintmk 64b9e6a
new example
martintmk c2dc417
cleanup
martintmk 4e9eb87
Update layer.rs
martintmk b999153
Update layer.rs
martintmk 32709e6
Update service.rs
martintmk File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| // Copyright (c) Microsoft Corporation. | ||
| // Licensed under the MIT License. | ||
|
|
||
| //! Chaos injection middleware example that injects faults with a configurable probability. | ||
| //! | ||
| //! This example simulates a service where 30% of requests are intercepted and | ||
| //! replaced with an injected error output, demonstrating how chaos injection can | ||
| //! be used to test resilience under failure conditions. | ||
|
|
||
| use layered::{Execute, Service, Stack}; | ||
| use seatbelt::ResilienceContext; | ||
| use seatbelt::chaos::injection::Injection; | ||
| use tick::Clock; | ||
|
|
||
| #[tokio::main] | ||
| async fn main() { | ||
| let clock = Clock::new_tokio(); | ||
| let context = ResilienceContext::new(&clock); | ||
|
|
||
| // Define stack with injection layer | ||
| let stack = ( | ||
| Injection::layer("my_injection", &context) | ||
| // Required: probability of injection (30%) | ||
| .rate(0.3) | ||
| // Required: the output to inject (receives the consumed input) | ||
| .output_with(|input: String, _args| format!("INJECTED_FAULT for '{input}'")), | ||
| Execute::new(execute_operation), | ||
| ); | ||
|
|
||
| // Create the service from the stack | ||
| let service = stack.into_service(); | ||
|
|
||
| for i in 0..10 { | ||
| let result = service.execute(format!("request-{i}")).await; | ||
| println!("{i}: result = '{result}'"); | ||
| } | ||
| } | ||
|
|
||
| async fn execute_operation(input: String) -> String { | ||
| format!("processed:{input}") | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| // Copyright (c) Microsoft Corporation. | ||
| // Licensed under the MIT License. | ||
|
|
||
| /// Arguments passed to the [`output_with`][super::InjectionLayer::output_with] callback. | ||
| /// | ||
| /// This type is `#[non_exhaustive]` so that additional fields can be added in the | ||
| /// future without a breaking change. | ||
| #[derive(Debug)] | ||
| #[non_exhaustive] | ||
| #[expect( | ||
| clippy::empty_structs_with_brackets, | ||
| reason = "non_exhaustive requires braces for forward-compatibility" | ||
| )] | ||
| pub struct InjectionOutputArgs {} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| // Copyright (c) Microsoft Corporation. | ||
| // Licensed under the MIT License. | ||
|
|
||
| use super::InjectionOutputArgs; | ||
|
|
||
| crate::utils::define_fn_wrapper!(InjectionOutput<In, Out>(Fn(In, InjectionOutputArgs) -> Out)); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| // Copyright (c) Microsoft Corporation. | ||
| // Licensed under the MIT License. | ||
|
|
||
| /// Default injection rate (no injection). | ||
| const DEFAULT_RATE: f64 = 0.0; | ||
|
|
||
| /// Configuration for the [`Injection`][super::Injection] middleware. | ||
| /// | ||
| /// This type can be deserialized from configuration files when the `serde` | ||
| /// feature is enabled. | ||
| /// | ||
| /// # Example | ||
| /// | ||
| /// ```rust | ||
| /// use seatbelt::chaos::injection::InjectionConfig; | ||
| /// | ||
| /// let config = InjectionConfig::default(); | ||
| /// assert!(config.enabled); | ||
| /// assert_eq!(config.rate, 0.0); | ||
| /// ``` | ||
| #[derive(Debug, Clone, PartialEq)] | ||
| #[cfg_attr(any(feature = "serde", test), derive(serde::Serialize, serde::Deserialize))] | ||
| #[non_exhaustive] | ||
| pub struct InjectionConfig { | ||
| /// Whether the injection middleware is enabled. When `false`, the middleware | ||
| /// is bypassed and requests pass through directly to the inner service. | ||
| pub enabled: bool, | ||
|
|
||
| /// The probability of injecting the configured output instead of calling the | ||
| /// inner service. Must be in the range `[0.0, 1.0]` where `0.0` means never | ||
| /// inject and `1.0` means always inject. | ||
| pub rate: f64, | ||
| } | ||
|
|
||
| impl Default for InjectionConfig { | ||
| fn default() -> Self { | ||
| Self { | ||
| enabled: true, | ||
| rate: DEFAULT_RATE, | ||
| } | ||
| } | ||
| } | ||
|
|
||
| #[cfg_attr(coverage_nightly, coverage(off))] | ||
| #[cfg(test)] | ||
| mod tests { | ||
| use super::*; | ||
|
|
||
| #[test] | ||
| #[cfg_attr(miri, ignore)] | ||
| fn default_snapshot() { | ||
| let config = InjectionConfig::default(); | ||
| insta::assert_json_snapshot!(config); | ||
| } | ||
|
|
||
| #[test] | ||
| fn serde_roundtrip() { | ||
| let config = InjectionConfig { | ||
| enabled: false, | ||
| rate: 0.42, | ||
| }; | ||
|
|
||
| let json = serde_json::to_string(&config).unwrap(); | ||
| let deserialized: InjectionConfig = serde_json::from_str(&json).unwrap(); | ||
|
|
||
| assert_eq!(config, deserialized); | ||
| } | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.