Skip to content

Commit 2e4aab4

Browse files
committed
more fixes
1 parent 942c3d0 commit 2e4aab4

11 files changed

+317
-46
lines changed

benchmarks/README.md

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
# Virtual Clock Performance Benchmarks
2+
3+
This directory contains performance benchmarks for the GenServerVirtualTime library, focusing on core virtual clock operations and actor simulation performance.
4+
5+
## Running Benchmarks
6+
7+
### Core Virtual Clock Benchmarks
8+
9+
Run the core virtual clock performance benchmarks:
10+
11+
```bash
12+
mix run benchmarks/virtual_clock_core_benchmarks.exs
13+
```
14+
15+
This benchmark suite measures:
16+
17+
- **VirtualClock.advance**: Core time advancement performance (1 second, 1 minute)
18+
- **VirtualClock.schedule_event**: Event scheduling performance (single and multiple events)
19+
- **Periodic ticking GenServer**: GenServer with virtual time that ticks every second
20+
- **ActorSimulation.simple**: Simple actor simulation with 10 events
21+
22+
### Expected Performance
23+
24+
Based on the latest benchmark results:
25+
26+
| Operation | Performance | Notes |
27+
|-----------|-------------|-------|
28+
| VirtualClock.advance (1 second) | ~248,270 ips | Very fast - pure time advancement |
29+
| VirtualClock.advance (1 minute) | ~221,228 ips | Similar performance for longer durations |
30+
| VirtualClock.schedule_event | ~929 ips | Includes event scheduling overhead |
31+
| Periodic ticking GenServer (5 ticks) | ~180 ips | GenServer callback execution |
32+
| ActorSimulation.simple (10 events) | ~93 ips | Full actor simulation |
33+
| VirtualClock.schedule_multiple_events | ~9 ips | 100 events scheduled and processed |
34+
35+
### Performance Analysis
36+
37+
The benchmarks reveal the performance characteristics:
38+
39+
1. **Pure time advancement is very fast** (~250k operations/second)
40+
2. **Event scheduling adds overhead** (~1ms per event)
41+
3. **GenServer callbacks are the bottleneck** (~5ms for 5 ticks)
42+
4. **ActorSimulation has additional overhead** (~11ms for 10 events)
43+
5. **Multiple events scale poorly** (~108ms for 100 events)
44+
45+
### Bottleneck Identification
46+
47+
The main bottleneck is in the virtual clock's advance mechanism when processing multiple events. The `Process.send_after(self(), {:do_advance, target_time, from}, 0)` call creates a real-time delay for each event, which accumulates for large numbers of events.
48+
49+
### Optimization Opportunities
50+
51+
1. **Batch event processing**: Process multiple events at the same time point in a single pass
52+
2. **Reduce Process.send_after overhead**: Use direct message passing where possible
53+
3. **Event scheduling optimization**: Use more efficient data structures for scheduled events
54+
55+
## Benchmark Results
56+
57+
The benchmarks generate console output showing:
58+
- **ips**: Iterations per second
59+
- **average**: Average execution time
60+
- **deviation**: Standard deviation
61+
- **median**: Median execution time
62+
- **99th %**: 99th percentile execution time
63+
- **Memory usage**: Memory consumption per operation
64+
65+
## Adding New Benchmarks
66+
67+
To add new benchmarks, edit `benchmarks/virtual_clock_core_benchmarks.exs`:
68+
69+
1. Add a new benchmark function
70+
2. Add it to the `Benchee.run/2` map
71+
3. Update the documentation
72+
73+
Example:
74+
75+
```elixir
76+
defp my_new_benchmark do
77+
# Your benchmark code here
78+
:ok
79+
end
80+
81+
# In the Benchee.run map:
82+
"My New Benchmark" => fn -> my_new_benchmark() end,
83+
```
84+
85+
## Dependencies
86+
87+
The benchmarks require:
88+
- `benchee` ~> 1.5 (added to mix.exs as dev dependency)
89+
- All GenServerVirtualTime modules
90+
91+
## Notes
92+
93+
- Benchmarks run with 5-second time limit and 1-second warmup
94+
- Memory usage is measured for each operation
95+
- Results are printed to console
96+
- All benchmarks include proper cleanup (stopping GenServers)
Lines changed: 167 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,167 @@
1+
# Virtual Clock Core Performance Benchmarks
2+
# Run with: mix run benchmarks/virtual_clock_core_benchmarks.exs
3+
#
4+
# These benchmarks focus on core virtual clock operations that should run
5+
# quickly and repeatedly to identify performance bottlenecks.
6+
7+
defmodule VirtualClockCoreBenchmarks do
8+
@moduledoc """
9+
Benchmarks for core virtual clock operations to identify performance bottlenecks.
10+
"""
11+
12+
def run_benchmarks do
13+
IO.puts("🚀 Starting Virtual Clock Core Performance Benchmarks...")
14+
IO.puts("=" |> String.duplicate(60))
15+
16+
Benchee.run(
17+
%{
18+
"VirtualClock.advance (1 second)" => fn -> advance_one_second() end,
19+
"VirtualClock.advance (1 minute)" => fn -> advance_one_minute() end,
20+
"VirtualClock.schedule_event" => fn -> schedule_single_event() end,
21+
"VirtualClock.schedule_multiple_events" => fn -> schedule_multiple_events() end,
22+
"Periodic ticking GenServer (5 ticks)" => fn -> periodic_ticking_benchmark() end,
23+
"ActorSimulation.simple (10 events)" => fn -> simple_actor_simulation() end
24+
},
25+
time: 5,
26+
memory_time: 2,
27+
warmup: 1,
28+
formatters: [
29+
Benchee.Formatters.Console
30+
]
31+
)
32+
33+
IO.puts("\n🏆 Core Benchmark Results:")
34+
IO.puts(" - VirtualClock.advance: Core time advancement performance")
35+
IO.puts(" - VirtualClock.schedule_event: Event scheduling performance")
36+
IO.puts(" - VirtualTimeGenServer: GenServer with virtual time performance")
37+
IO.puts(" - ActorSimulation: Simple actor simulation performance")
38+
IO.puts("\n📊 Check benchmarks/results/ for detailed HTML reports")
39+
end
40+
41+
# Benchmark: VirtualClock.advance for 1 second
42+
defp advance_one_second do
43+
{:ok, clock} = VirtualClock.start_link()
44+
VirtualClock.advance(clock, 1000)
45+
GenServer.stop(clock)
46+
:ok
47+
end
48+
49+
# Benchmark: VirtualClock.advance for 1 minute
50+
defp advance_one_minute do
51+
{:ok, clock} = VirtualClock.start_link()
52+
VirtualClock.advance(clock, 60_000)
53+
GenServer.stop(clock)
54+
:ok
55+
end
56+
57+
58+
# Benchmark: Schedule a single event
59+
defp schedule_single_event do
60+
{:ok, clock} = VirtualClock.start_link()
61+
VirtualClock.send_after(clock, self(), :test_message, 1000)
62+
VirtualClock.advance(clock, 1000)
63+
GenServer.stop(clock)
64+
:ok
65+
end
66+
67+
# Benchmark: Schedule multiple events
68+
defp schedule_multiple_events do
69+
{:ok, clock} = VirtualClock.start_link()
70+
71+
# Schedule 100 events at different times
72+
for i <- 1..100 do
73+
VirtualClock.send_after(clock, self(), {:test_message, i}, i * 10)
74+
end
75+
76+
VirtualClock.advance(clock, 1000)
77+
GenServer.stop(clock)
78+
:ok
79+
end
80+
81+
82+
# Benchmark: Periodic ticking GenServer with advance time
83+
defp periodic_ticking_benchmark do
84+
{:ok, clock} = VirtualClock.start_link()
85+
86+
# Start a periodic ticking GenServer
87+
{:ok, server} = VirtualTimeGenServer.start_link(PeriodicTicker, %{}, virtual_clock: clock)
88+
89+
# Advance time by 5 seconds (should trigger 5 ticks)
90+
VirtualClock.advance(clock, 5000)
91+
92+
# Get the tick count
93+
count = VirtualTimeGenServer.call(server, :get_tick_count)
94+
95+
# Assert we got the expected number of ticks
96+
assert count >= 5, "Expected at least 5 ticks, got #{count}"
97+
98+
GenServer.stop(server)
99+
GenServer.stop(clock)
100+
count
101+
end
102+
103+
# Benchmark: Simple actor simulation
104+
defp simple_actor_simulation do
105+
simulation =
106+
ActorSimulation.new()
107+
|> ActorSimulation.add_actor(:sender,
108+
send_pattern: {:periodic, 100, :message},
109+
targets: [:receiver]
110+
)
111+
|> ActorSimulation.add_actor(:receiver)
112+
|> ActorSimulation.run(duration: 1000) # 1 second simulation
113+
114+
ActorSimulation.stop(simulation)
115+
:ok
116+
end
117+
118+
defp assert(condition, message \\ "Assertion failed") do
119+
unless condition do
120+
raise ArgumentError, message
121+
end
122+
end
123+
end
124+
125+
# Test server modules for benchmarking
126+
defmodule BenchmarkTestServer do
127+
use VirtualTimeGenServer
128+
129+
def init(_args) do
130+
{:ok, %{count: 0}}
131+
end
132+
133+
def handle_call(:increment, _from, state) do
134+
{:reply, :ok, %{state | count: state.count + 1}}
135+
end
136+
137+
def handle_call(:get_count, _from, state) do
138+
{:reply, state.count, state}
139+
end
140+
end
141+
142+
defmodule PeriodicTicker do
143+
use VirtualTimeGenServer
144+
145+
def init(_args) do
146+
# Schedule the first tick
147+
VirtualTimeGenServer.send_after(self(), :tick, 1000)
148+
{:ok, %{tick_count: 0}}
149+
end
150+
151+
def handle_info(:tick, state) do
152+
# Increment tick count and schedule next tick
153+
new_state = %{state | tick_count: state.tick_count + 1}
154+
VirtualTimeGenServer.send_after(self(), :tick, 1000)
155+
{:noreply, new_state}
156+
end
157+
158+
def handle_call(:get_tick_count, _from, state) do
159+
{:reply, state.tick_count, state}
160+
end
161+
end
162+
163+
# Create results directory
164+
File.mkdir_p("benchmarks/results")
165+
166+
# Run the benchmarks
167+
VirtualClockCoreBenchmarks.run_benchmarks()

lib/virtual_clock.ex

Lines changed: 32 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -153,6 +153,19 @@ defmodule VirtualClock do
153153

154154
@impl true
155155
def handle_info({:do_advance, target_time, from}, state) do
156+
advance_loop(state, target_time, from)
157+
end
158+
159+
# Efficient advance loop that processes all events without real-time delays
160+
defp advance_loop(state, target_time, from) do
161+
# Process events in chronological order, allowing for newly scheduled events
162+
# Use a loop to avoid stack overflow
163+
advance_loop_iterative(state, target_time, from)
164+
end
165+
166+
defp advance_loop_iterative(state, target_time, from) do
167+
# Process events in batches to allow other processes to proceed
168+
# This ensures that actors can process messages and schedule new ones
156169
case find_next_event_time_up_to(state.scheduled, target_time) do
157170
nil ->
158171
# No more events, finish advance
@@ -161,43 +174,28 @@ defmodule VirtualClock do
161174
{:noreply, new_state}
162175

163176
next_time ->
164-
# Trigger all events at this exact time
177+
# Process all events up to the next time point
165178
{triggered, remaining} = split_events_at_time(state.scheduled, next_time)
166179

167180
Enum.each(triggered, fn event ->
168181
send(event.dest, event.message)
169182
end)
170183

171184
# Update state and continue advancing
172-
# Add a tiny delay to allow other processes to handle messages
173185
new_state = %{state | current_time: next_time, scheduled: remaining}
174186

175-
# Schedule continuation with a tiny delay for message processing
176-
# The delay allows triggered processes to handle their messages
177-
# and make new send_after calls that we'll process
178-
:erlang.send_after(0, self(), {:do_advance, target_time, from})
187+
# Continue advancing by sending message to self
188+
# This allows other processes to proceed
189+
# Use a tiny delay to allow processes to handle messages and schedule new ones
190+
Process.send_after(self(), {:do_advance, target_time, from}, 0)
179191
{:noreply, new_state}
180192
end
181193
end
182194

183-
# Private helpers
184-
185-
defp split_events(scheduled, time) do
186-
Enum.split_with(scheduled, fn event -> event.trigger_time <= time end)
187-
end
188-
189195
defp split_events_at_time(scheduled, time) do
190196
Enum.split_with(scheduled, fn event -> event.trigger_time == time end)
191197
end
192198

193-
defp find_next_event_time([]), do: nil
194-
195-
defp find_next_event_time(scheduled) do
196-
scheduled
197-
|> Enum.map(& &1.trigger_time)
198-
|> Enum.min()
199-
end
200-
201199
defp find_next_event_time_up_to([], _target), do: nil
202200

203201
defp find_next_event_time_up_to(scheduled, target_time) do
@@ -208,4 +206,18 @@ defmodule VirtualClock do
208206
events -> events |> Enum.map(& &1.trigger_time) |> Enum.min()
209207
end
210208
end
209+
210+
# Private helpers
211+
212+
defp split_events(scheduled, time) do
213+
Enum.split_with(scheduled, fn event -> event.trigger_time <= time end)
214+
end
215+
216+
defp find_next_event_time([]), do: nil
217+
218+
defp find_next_event_time(scheduled) do
219+
scheduled
220+
|> Enum.map(& &1.trigger_time)
221+
|> Enum.min()
222+
end
211223
end

lib/virtual_time_gen_state_machine.ex

Lines changed: 2 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -194,12 +194,7 @@ defmodule VirtualTimeGenStateMachine do
194194
Process.put(:time_backend, final_backend)
195195

196196
# Call the module's init function
197-
case module.init(init_arg) do
198-
{:ok, state, data} -> {:ok, state, data}
199-
{:ok, state, data, timeout} -> {:ok, state, data, timeout}
200-
{:ok, state, data, {:continue, arg}} -> {:ok, state, data, {:continue, arg}}
201-
other -> other
202-
end
197+
module.init(init_arg)
203198
end
204199

205200
# Start with a wrapper that injects the virtual clock
@@ -337,12 +332,7 @@ defmodule VirtualTimeGenStateMachine.Wrapper do
337332
Process.put(:__vtgsm_module__, module)
338333

339334
# Call the init function which injects the virtual clock
340-
case init_fun.() do
341-
{:ok, state, data} -> {:ok, state, data}
342-
{:ok, state, data, timeout} -> {:ok, state, data, timeout}
343-
{:ok, state, data, {:continue, arg}} -> {:ok, state, data, {:continue, arg}}
344-
other -> other
345-
end
335+
init_fun.()
346336
end
347337

348338
def handle_event(event_type, event_content, state, data) do

mix.exs

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -82,7 +82,10 @@ defmodule GenServerVirtualTime.MixProject do
8282

8383
# Mutation testing
8484
{:muzak, "~> 1.1", only: :test, runtime: false},
85-
{:exavier, "~> 0.3.0", only: :test, runtime: false}
85+
{:exavier, "~> 0.3.0", only: :test, runtime: false},
86+
87+
# Benchmarking
88+
{:benchee, "~> 1.5", only: :dev, runtime: false}
8689
]
8790
end
8891

0 commit comments

Comments
 (0)