Flaky test: CircuitBreakerSpec.ASynchronousCircuitBreakerThatIsOpen throws TimeoutException instead of TestException

[Aaronontheweb]

Version Information
Version of Akka.NET? Latest dev branch
Which Akka.NET Modules? Akka (core)

Describe the bug
The test Akka.Tests.Pattern.ASynchronousCircuitBreakerThatIsOpen.A synchronous circuit breaker that is open must throw exceptions when called before reset timeout fails intermittently in CI. The test expects a TestException to be thrown, but receives a TimeoutException instead.

To Reproduce
Steps to reproduce the behavior:

1. Run dotnet test src/core/Akka.Tests/Akka.Tests.csproj -c Release --filter FullyQualifiedName~ASynchronousCircuitBreakerThatIsOpen.Must_throw_exceptions_before_reset_timeout
2. Test passes locally but fails intermittently in CI environment

This appears to be a timing-dependent issue where the circuit breaker's 100ms timeout is being exceeded before the test exception can be thrown.

Expected behavior
The test should throw Akka.Tests.Pattern.TestException when the circuit breaker is called while open.

Actual behavior
The test throws System.TimeoutException with message "Execution did not complete within the time allotted 100 ms" instead of the expected TestException.

Full stack trace:

Assert.ThrowsAny() Failure: Exception type was not compatible
Expected: typeof(Akka.Tests.Pattern.TestException)
Actual:   typeof(System.TimeoutException)
---- System.TimeoutException : Execution did not complete within the time allotted 100 ms

   at Akka.Tests.Pattern.ASynchronousCircuitBreakerThatIsOpen.Must_throw_exceptions_before_reset_timeout() in D:\a\1\s\src\core\Akka.Tests\Pattern\CircuitBreakerSpec.cs:line 161
----- Inner Stack Trace -----
   at PolyfillExtensions.<WaitAsync>d__84.MoveNext() in C:\Users\VssAdministrator\.nuget\packages\polyfill\1.28.0\contentFiles\cs\netstandard2.0\Polyfill\PolyfillExtensions_Task.cs:line 49
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at PolyfillExtensions.<WaitAsync>d__83.MoveNext() in C:\Users\VssAdministrator\.nuget\packages\polyfill\1.28.0\contentFiles\cs\netstandard2.0\Polyfill\PolyfillExtensions_Task.cs:line 27
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Akka.Util.Internal.AtomicState.<CallThrough>d__14`1.MoveNext() in D:\a\1\s\src\core\Akka\Util\Internal\AtomicState.cs:line 189
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Akka.Util.Internal.AtomicState.<CallThrough>d__14`1.MoveNext() in D:\a\1\s\src\core\Akka\Util\Internal\AtomicState.cs:line 196
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Akka.Pattern.CircuitBreaker.WithSyncCircuitBreaker(Action body) in D:\a\1\s\src\core\Akka\Pattern\CircuitBreaker.cs:line 301

Environment

• OS: CI environment (Windows Azure DevOps runner, .NET Framework 4.8)
• .NET Version: .NET Framework 4.8
• Framework: net48

Related Issues and PRs

• Alternative implementation of AtomicState leveraging WaitAsync #6109 - Alternative implementation of AtomicState leveraging WaitAsync (merged 2023-03-28, milestone 1.5.2) - Changed circuit breaker to use Task.WaitAsync which can throw TimeoutException
• Port Akka.Tests.Pattern tests to async/await - CircuitBreakerSpec #5830 - Port Akka.Tests.Pattern tests to async/await - CircuitBreakerSpec (merged 2022-04-22)

Additional context

• The 100ms CallTimeout is very tight and may be exceeded in CI environments under load
• The AtomicState.CallThrough method uses WaitAsync which throws TimeoutException when the timeout is exceeded before the underlying task completes
• This is a race between the test throwing TestException and the circuit breaker's 100ms timeout expiring
• May require increasing the timeout or modifying the test to account for both exception types
• Test file: src/core/Akka.Tests/Pattern/CircuitBreakerSpec.cs:161

[Aaronontheweb]

This test failure also occurred on PR #7949 (unrelated changes to sharding delivery tests).

Build: Azure DevOps Build 121497
Job: .NET Unit Tests (Windows)
Date: 2025-11-25

This confirms the test is intermittently failing on Windows in CI, independent of code changes to other modules.