|
1 | 1 | --- |
2 | 2 | layout: docs |
3 | 3 | title: Overview |
| 4 | +section: "Section 1" |
4 | 5 | --- |
5 | | -# Nighthawk: architecture and key concepts |
6 | 6 |
|
7 | | -## High level interaction model |
8 | | - |
9 | | -**Process** creates one or more **Workers**. **Worker** will run **Sequencer**, |
10 | | -which in turn queries **RateLimiter** for [request-release |
11 | | -timings](terminology.md#request-release-timings). When it is time to release a |
12 | | -request, **BenchmarkClient** will be requested to do so by **Sequencer**. |
13 | | -**BenchMarkClient** will then ask its underlying **Pool** to create a |
14 | | -**StreamDecoder** for releasing the actual request. **StreamDecoder** will query the |
15 | | -request data it needs to send from the configured **RequestSource**, and send it |
16 | | -off. **StreamDecoder** will emit events as it progresses (pool ready, |
17 | | -completion, etc), and timings will subsequently be recorded into **Statistic** |
18 | | -as well as get bubbled up to **Sequencer** for tracking in-flight work and |
19 | | -**Statistic** bookkeeping. |
20 | | - |
21 | | -**Sequencer** will query the configured **TerminationPredicates** to terminate |
22 | | -when and how to terminate execution. When all **Workers** have finished, |
23 | | -**Process** will collect the results from all **Workers** via |
24 | | -**OutputCollector**, and run **OutputFormatter** to transform to the requested |
25 | | -output format. |
26 | | - |
27 | | -## Notable upcoming changes |
28 | | - |
29 | | -Calling out two new concepts that may get proposed in the future, and cause some |
30 | | -churn in the code base as we inject them. |
31 | | - |
32 | | -### Phases |
33 | | - |
34 | | -One notable addition / change that may get proposed in the near future is the |
35 | | -introduction of **Phase**. **Phase** would represent a distinct stage of an |
36 | | -execution, for example a warm-up. It would then be useful to have |
37 | | -per-phase reporting of latencies as well as counters and latencies. |
38 | | - |
39 | | -Concretely, a warm-up phase could be represented by a distinct duration |
40 | | -termination predicate and a ramping rate limiter. Upon completion, the `hot` |
41 | | -BenchmarkClient with its associated pool would then be live-transferred to the |
42 | | -next configured phase, after which execution can continue. |
43 | | - |
44 | | -One other reason to have this is that it would enable remote- and/or cli- |
45 | | -controlled ramping of certain test parameters by associating those to different |
46 | | -phases. Termination predicates can be leveraged to immediately terminate the |
47 | | -current phase after injecting a new one, allowing for real-time steering via |
48 | | -gRPC and/or CLI. |
49 | | - |
50 | | -### Streaming parameterization and output stats |
51 | | - |
52 | | -Once we have phases, the gRPC service, and perhaps the CLI, would be natural |
53 | | -candidates to follow up with to allow dynamic phase injection, as well as send |
54 | | -back reports per phase. |
55 | | - |
56 | | -## Key concept descriptions |
57 | | - |
58 | | -*The c++ interface definitions for the concepts below can be found [here](https://github.com/envoyproxy/nighthawk/tree/main/include/nighthawk)*. |
59 | | - |
60 | | -### Process |
61 | | - |
62 | | -**Process** represents the primary entry point to a Nighthawk execution run. |
63 | | -Only one Process is allowed to exist at the same point in time within an OS |
64 | | -Process. **Process** is responsible for performing process-wide initialization |
65 | | -and termination, as well as handle input configuration, deliver output, and |
66 | | -co-ordinate Workers. **ProcessImpl** is re-used across the CLI and the gRPC |
67 | | -service. |
68 | | - |
69 | | -### Worker |
70 | | - |
71 | | -**Worker** is responsible for performing correct initialization and termination |
72 | | -of its thread, as well as execution of its designated task and offering a way |
73 | | -for consumers to wait for that task to complete. |
74 | | - |
75 | | -### TerminationPredicate |
76 | | - |
77 | | -**TerminationPredicate** is responsible for if and how to terminate. As of |
78 | | -today, there are two types: one that will indicate that it is time to terminate |
79 | | -based on a pre-configured duration, and one that will do so based on absolute |
80 | | -counter thresholds. |
81 | | - |
82 | | -### Sequencer |
83 | | - |
84 | | -**SequencerImpl** resides on a worker-thread, and drives itself via timers that |
85 | | -run on the dispatcher, and coordinates interaction between **RateLimiter,** |
86 | | -**BenchmarkClient**, and **TerminationPredicate** to drive execution to |
87 | | -completion. |
88 | | - |
89 | | -### RateLimiter |
90 | | - |
91 | | -**RateLimiter** is responsible for indicating when it is time to release a |
92 | | -request. **RateLimiter** offers a semaphore-like interaction model, as in |
93 | | -[closed-loop](terminology.md#closed-loop) mode it may be that |
94 | | -**BenchmarkClient** is not able to satisfy request-release timings, in which |
95 | | -case acquisitions from **RateLimiter** need to be cancelled. Concretely, as of |
96 | | -today there is **LinearRateLimiterImpl** which offers a straight-paced plain |
97 | | -frequency, as well as work in progress on |
98 | | -**DistributionSamplingRateLimiterImpl** (adding uniformly distributed random |
99 | | -timing offsets to an underlying **RateLimiter**) and **LinearRampingRateLimiter**. |
100 | | - |
101 | | -### BenchmarkClient |
102 | | - |
103 | | -As of today, there’s a single implementation called **BenchmarkClientImpl**, |
104 | | -which wraps Envoy’s **Upstream** concept and (slightly) customized H1/H2 |
105 | | -**Pool** concepts. For executing requests, the pool will be requested to create |
106 | | -a **StreamEncoder**, and Nighthawk will pass its own **StreamDecoderImpl** into |
107 | | -that as an argument. The integration surface between **BenchmarkClient** is |
108 | | -defined via `BenchmarkClient::tryStartRequest()` and a callback specification |
109 | | -which will be fired upon completion of a successfully started request. |
110 | | - |
111 | | -For H3, it is anticipated that it will fit into this model, but if all else |
112 | | -fails, it will be entirely possible to wire in a new type of |
113 | | -**BenchmarkClient**. |
114 | | - |
115 | | -### RequestSource |
116 | | - |
117 | | -**RequestSource** is an abstraction that allows us to implement different ways |
118 | | -for **BenchmarkClient** to get information on what the request that it is about |
119 | | -to fire off should look like. Today, two implementations exist: a static one, |
120 | | -which will repeat the same request over and over, as well as one that pulls |
121 | | -dynamic request data from a grpc service. The latter can, for example, be used |
122 | | -to implement log-replay. |
123 | | - |
124 | | -### StreamDecoder |
125 | | - |
126 | | -**StreamDecoder** is a Nighthawk-specific implementation of an [Envoy |
127 | | -concept](https://github.com/envoyproxy/envoy/blob/3156229006a5340b65c773329070737f67e81826/include/envoy/http/filter.h#L463). |
128 | | -StreamDecoder will by notified by Envoy as headers and body fragments arrive. |
129 | | -The Nighthawk implementation of that is responsible for coordinating lifetime |
130 | | -events of a request to upper abstraction layers (**BenchmarkClient**, |
131 | | -**Sequencer**) as well as recording latency and reporting that upwards. |
132 | | - |
133 | | -### OutputCollector |
134 | | - |
135 | | -**OutputCollector** is a container that facilitates building up the native output |
136 | | -format of Nighthawk (`proto3`, `nighthawk::client::Output`). It is the basis for all |
137 | | -output formats offered by Nighthawk, including CLI human output. |
138 | | - |
139 | | -### OutputFormatter |
140 | | - |
141 | | -**OutputFormatter** is responsible for transformations of `nighthawk::client::Output` |
142 | | -to requested formats (e.g. human, json, fortio, etc) |
143 | | - |
144 | | -### Statistic |
145 | | - |
146 | | -Nighthawk’s **Statistic** is responsible for administrating latencies. The most |
147 | | -notable implementation that exists today wraps |
148 | | -[HdrHistogram](https://github.com/HdrHistogram/HdrHistogram_c), but Nighthawk |
149 | | -also has a couple of other implementations which mostly exist to ensure that |
150 | | -floating point math is correct in tests, as well as a simple efficient |
151 | | -implementation that simply tracks the `mean` and `pstddev` for those cases where |
152 | | -we don't need percentiles. For various reasons, HdrHistogram might get replaced |
153 | | -by [libcirclhist](https://github.com/envoyproxy/nighthawk/issues/115) in the |
154 | | -near future. |
155 | | - |
156 | | -### H1Pool & H2Pool |
157 | | - |
158 | | -Nighthawk derives its own version of these from the vanilla Envoy ones. It does |
159 | | -that to implement things like pro-active connection pre-fetching and H2 |
160 | | -multi-connection support, as well as offer more connection management |
161 | | -strategies. |
162 | | - |
163 | | -## Nighthawk binaries |
164 | | - |
165 | | -### nighthawk_client |
166 | | - |
167 | | -The CLI interface of the Nighthawk client. It synthesizes traffic according |
168 | | -to the requested configuration and report results in the requested output format. |
169 | | - |
170 | | -### nighthawk_service |
171 | | - |
172 | | -Nighthawk’s gRPC service is able to execute load tests, and report results. |
173 | | -Under the hood it shares much of the code of nighthawk_client, and effectively |
174 | | -it allows to efficiently perform remote back-to-back executions of that. |
175 | | - |
176 | | -### nighthawk_test_server |
177 | | - |
178 | | -Nighthawk’s test server, based on Envoy. It is able to synthesize delays and |
179 | | -responses based on configuration via request headers (next to on-disk |
180 | | -configuration). |
181 | | - |
182 | | -### nighthawk_output_transform |
183 | | - |
184 | | -Utility for transforming the nighthawk-native json output format into |
185 | | -other formats (e.g. human, fortio). It can be very useful to always store the |
186 | | -json output format, yet be able to easily get to one of the other output |
187 | | -formats. It’s like having the cake and eating it too! |
188 | | - |
189 | | -## User-specified Nighthawk logging |
190 | | - |
191 | | -Users of Nighthawk can specify custom format and destination (logging sink |
192 | | -delegate) for all Nighthawk logging messages. Nighthawk utilizes the Envoy's |
193 | | -logging mechanism by performing all logging via the **ENVOY_LOG** macro. To |
194 | | -customize this mechanism, users need to perform two steps: |
195 | | -1. Create a logging sink delegate inherited from [Envoy SinkDelegate](https://github.com/envoyproxy/envoy/blob/main/source/common/common/logger.h). |
196 | | -2. Construct a ServiceImpl object with an [Envoy Logger Context](https://github.com/envoyproxy/envoy/blob/main/source/common/common/logger.h) which contains user-specified log level and format. |
| 7 | +Test section 1 |
0 commit comments