|
1 | 1 | --- |
2 | | -layout: doc |
3 | | -title: Using GetNighthawk |
| 2 | +layout: docs |
| 3 | +title: Overview |
4 | 4 | --- |
5 | | -Test 2 |
| 5 | +# Nighthawk: architecture and key concepts |
| 6 | + |
| 7 | +## High level interaction model |
| 8 | + |
| 9 | +**Process** creates one or more **Workers**. **Worker** will run **Sequencer**, |
| 10 | +which in turn queries **RateLimiter** for [request-release |
| 11 | +timings](terminology.md#request-release-timings). When it is time to release a |
| 12 | +request, **BenchmarkClient** will be requested to do so by **Sequencer**. |
| 13 | +**BenchMarkClient** will then ask its underlying **Pool** to create a |
| 14 | +**StreamDecoder** for releasing the actual request. **StreamDecoder** will query the |
| 15 | +request data it needs to send from the configured **RequestSource**, and send it |
| 16 | +off. **StreamDecoder** will emit events as it progresses (pool ready, |
| 17 | +completion, etc), and timings will subsequently be recorded into **Statistic** |
| 18 | +as well as get bubbled up to **Sequencer** for tracking in-flight work and |
| 19 | +**Statistic** bookkeeping. |
| 20 | + |
| 21 | +**Sequencer** will query the configured **TerminationPredicates** to terminate |
| 22 | +when and how to terminate execution. When all **Workers** have finished, |
| 23 | +**Process** will collect the results from all **Workers** via |
| 24 | +**OutputCollector**, and run **OutputFormatter** to transform to the requested |
| 25 | +output format. |
| 26 | + |
| 27 | +## Notable upcoming changes |
| 28 | + |
| 29 | +Calling out two new concepts that may get proposed in the future, and cause some |
| 30 | +churn in the code base as we inject them. |
| 31 | + |
| 32 | +### Phases |
| 33 | + |
| 34 | +One notable addition / change that may get proposed in the near future is the |
| 35 | +introduction of **Phase**. **Phase** would represent a distinct stage of an |
| 36 | +execution, for example a warm-up. It would then be useful to have |
| 37 | +per-phase reporting of latencies as well as counters and latencies. |
| 38 | + |
| 39 | +Concretely, a warm-up phase could be represented by a distinct duration |
| 40 | +termination predicate and a ramping rate limiter. Upon completion, the `hot` |
| 41 | +BenchmarkClient with its associated pool would then be live-transferred to the |
| 42 | +next configured phase, after which execution can continue. |
| 43 | + |
| 44 | +One other reason to have this is that it would enable remote- and/or cli- |
| 45 | +controlled ramping of certain test parameters by associating those to different |
| 46 | +phases. Termination predicates can be leveraged to immediately terminate the |
| 47 | +current phase after injecting a new one, allowing for real-time steering via |
| 48 | +gRPC and/or CLI. |
| 49 | + |
| 50 | +### Streaming parameterization and output stats |
| 51 | + |
| 52 | +Once we have phases, the gRPC service, and perhaps the CLI, would be natural |
| 53 | +candidates to follow up with to allow dynamic phase injection, as well as send |
| 54 | +back reports per phase. |
| 55 | + |
| 56 | +## Key concept descriptions |
| 57 | + |
| 58 | +*The c++ interface definitions for the concepts below can be found [here](https://github.com/envoyproxy/nighthawk/tree/main/include/nighthawk)*. |
| 59 | + |
| 60 | +### Process |
| 61 | + |
| 62 | +**Process** represents the primary entry point to a Nighthawk execution run. |
| 63 | +Only one Process is allowed to exist at the same point in time within an OS |
| 64 | +Process. **Process** is responsible for performing process-wide initialization |
| 65 | +and termination, as well as handle input configuration, deliver output, and |
| 66 | +co-ordinate Workers. **ProcessImpl** is re-used across the CLI and the gRPC |
| 67 | +service. |
| 68 | + |
| 69 | +### Worker |
| 70 | + |
| 71 | +**Worker** is responsible for performing correct initialization and termination |
| 72 | +of its thread, as well as execution of its designated task and offering a way |
| 73 | +for consumers to wait for that task to complete. |
| 74 | + |
| 75 | +### TerminationPredicate |
| 76 | + |
| 77 | +**TerminationPredicate** is responsible for if and how to terminate. As of |
| 78 | +today, there are two types: one that will indicate that it is time to terminate |
| 79 | +based on a pre-configured duration, and one that will do so based on absolute |
| 80 | +counter thresholds. |
| 81 | + |
| 82 | +### Sequencer |
| 83 | + |
| 84 | +**SequencerImpl** resides on a worker-thread, and drives itself via timers that |
| 85 | +run on the dispatcher, and coordinates interaction between **RateLimiter,** |
| 86 | +**BenchmarkClient**, and **TerminationPredicate** to drive execution to |
| 87 | +completion. |
| 88 | + |
| 89 | +### RateLimiter |
| 90 | + |
| 91 | +**RateLimiter** is responsible for indicating when it is time to release a |
| 92 | +request. **RateLimiter** offers a semaphore-like interaction model, as in |
| 93 | +[closed-loop](terminology.md#closed-loop) mode it may be that |
| 94 | +**BenchmarkClient** is not able to satisfy request-release timings, in which |
| 95 | +case acquisitions from **RateLimiter** need to be cancelled. Concretely, as of |
| 96 | +today there is **LinearRateLimiterImpl** which offers a straight-paced plain |
| 97 | +frequency, as well as work in progress on |
| 98 | +**DistributionSamplingRateLimiterImpl** (adding uniformly distributed random |
| 99 | +timing offsets to an underlying **RateLimiter**) and **LinearRampingRateLimiter**. |
| 100 | + |
| 101 | +### BenchmarkClient |
| 102 | + |
| 103 | +As of today, there’s a single implementation called **BenchmarkClientImpl**, |
| 104 | +which wraps Envoy’s **Upstream** concept and (slightly) customized H1/H2 |
| 105 | +**Pool** concepts. For executing requests, the pool will be requested to create |
| 106 | +a **StreamEncoder**, and Nighthawk will pass its own **StreamDecoderImpl** into |
| 107 | +that as an argument. The integration surface between **BenchmarkClient** is |
| 108 | +defined via `BenchmarkClient::tryStartRequest()` and a callback specification |
| 109 | +which will be fired upon completion of a successfully started request. |
| 110 | + |
| 111 | +For H3, it is anticipated that it will fit into this model, but if all else |
| 112 | +fails, it will be entirely possible to wire in a new type of |
| 113 | +**BenchmarkClient**. |
| 114 | + |
| 115 | +### RequestSource |
| 116 | + |
| 117 | +**RequestSource** is an abstraction that allows us to implement different ways |
| 118 | +for **BenchmarkClient** to get information on what the request that it is about |
| 119 | +to fire off should look like. Today, two implementations exist: a static one, |
| 120 | +which will repeat the same request over and over, as well as one that pulls |
| 121 | +dynamic request data from a grpc service. The latter can, for example, be used |
| 122 | +to implement log-replay. |
| 123 | + |
| 124 | +### StreamDecoder |
| 125 | + |
| 126 | +**StreamDecoder** is a Nighthawk-specific implementation of an [Envoy |
| 127 | +concept](https://github.com/envoyproxy/envoy/blob/3156229006a5340b65c773329070737f67e81826/include/envoy/http/filter.h#L463). |
| 128 | +StreamDecoder will by notified by Envoy as headers and body fragments arrive. |
| 129 | +The Nighthawk implementation of that is responsible for coordinating lifetime |
| 130 | +events of a request to upper abstraction layers (**BenchmarkClient**, |
| 131 | +**Sequencer**) as well as recording latency and reporting that upwards. |
| 132 | + |
| 133 | +### OutputCollector |
| 134 | + |
| 135 | +**OutputCollector** is a container that facilitates building up the native output |
| 136 | +format of Nighthawk (`proto3`, `nighthawk::client::Output`). It is the basis for all |
| 137 | +output formats offered by Nighthawk, including CLI human output. |
| 138 | + |
| 139 | +### OutputFormatter |
| 140 | + |
| 141 | +**OutputFormatter** is responsible for transformations of `nighthawk::client::Output` |
| 142 | +to requested formats (e.g. human, json, fortio, etc) |
| 143 | + |
| 144 | +### Statistic |
| 145 | + |
| 146 | +Nighthawk’s **Statistic** is responsible for administrating latencies. The most |
| 147 | +notable implementation that exists today wraps |
| 148 | +[HdrHistogram](https://github.com/HdrHistogram/HdrHistogram_c), but Nighthawk |
| 149 | +also has a couple of other implementations which mostly exist to ensure that |
| 150 | +floating point math is correct in tests, as well as a simple efficient |
| 151 | +implementation that simply tracks the `mean` and `pstddev` for those cases where |
| 152 | +we don't need percentiles. For various reasons, HdrHistogram might get replaced |
| 153 | +by [libcirclhist](https://github.com/envoyproxy/nighthawk/issues/115) in the |
| 154 | +near future. |
| 155 | + |
| 156 | +### H1Pool & H2Pool |
| 157 | + |
| 158 | +Nighthawk derives its own version of these from the vanilla Envoy ones. It does |
| 159 | +that to implement things like pro-active connection pre-fetching and H2 |
| 160 | +multi-connection support, as well as offer more connection management |
| 161 | +strategies. |
| 162 | + |
| 163 | +## Nighthawk binaries |
| 164 | + |
| 165 | +### nighthawk_client |
| 166 | + |
| 167 | +The CLI interface of the Nighthawk client. It synthesizes traffic according |
| 168 | +to the requested configuration and report results in the requested output format. |
| 169 | + |
| 170 | +### nighthawk_service |
| 171 | + |
| 172 | +Nighthawk’s gRPC service is able to execute load tests, and report results. |
| 173 | +Under the hood it shares much of the code of nighthawk_client, and effectively |
| 174 | +it allows to efficiently perform remote back-to-back executions of that. |
| 175 | + |
| 176 | +### nighthawk_test_server |
| 177 | + |
| 178 | +Nighthawk’s test server, based on Envoy. It is able to synthesize delays and |
| 179 | +responses based on configuration via request headers (next to on-disk |
| 180 | +configuration). |
| 181 | + |
| 182 | +### nighthawk_output_transform |
| 183 | + |
| 184 | +Utility for transforming the nighthawk-native json output format into |
| 185 | +other formats (e.g. human, fortio). It can be very useful to always store the |
| 186 | +json output format, yet be able to easily get to one of the other output |
| 187 | +formats. It’s like having the cake and eating it too! |
| 188 | + |
| 189 | +## User-specified Nighthawk logging |
| 190 | + |
| 191 | +Users of Nighthawk can specify custom format and destination (logging sink |
| 192 | +delegate) for all Nighthawk logging messages. Nighthawk utilizes the Envoy's |
| 193 | +logging mechanism by performing all logging via the **ENVOY_LOG** macro. To |
| 194 | +customize this mechanism, users need to perform two steps: |
| 195 | +1. Create a logging sink delegate inherited from [Envoy SinkDelegate](https://github.com/envoyproxy/envoy/blob/main/source/common/common/logger.h). |
| 196 | +2. Construct a ServiceImpl object with an [Envoy Logger Context](https://github.com/envoyproxy/envoy/blob/main/source/common/common/logger.h) which contains user-specified log level and format. |
0 commit comments