Skip to content

Commit 5d61408

Browse files
bmorelli25beniwohli
authored andcommitted
docs: add frame context to tuning (#311)
1 parent 5895afb commit 5d61408

File tree

1 file changed

+26
-11
lines changed

1 file changed

+26
-11
lines changed

docs/tuning.asciidoc

Lines changed: 26 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
== Tuning and Overhead considerations
33

44
Using an APM solution comes with certain trade-offs, and the Python agent for Elastic APM is no different.
5-
Instrumenting your code, measuring timings, recording context data etc. all need resources:
5+
Instrumenting your code, measuring timings, recording context data, etc., all need resources:
66

77
* CPU time
88
* memory
@@ -16,11 +16,10 @@ But because every deployment is different, there are some knobs you can turn to
1616
[[tuning-sample-rate]]
1717
=== Transaction Sample Rate
1818

19-
The most straight forward way to reduce the overhead of the agent is to tell the agent to do less.
19+
The easiest way to reduce the overhead of the agent is to tell the agent to do less.
2020
If you set the <<config-transaction-sample-rate,`transaction_sample_rate`>> to a value below `1.0`,
2121
the agent will randomly sample only a subset of transactions.
22-
If a transaction is not sampled, the agent has to do a lot less work,
23-
as we only record the the name of the transaction, the overall transaction time and the result for unsampled transactions.
22+
Unsampled transactions only record the name of the transaction, the overall transaction time, and the result:
2423

2524
[options="header"]
2625
|============
@@ -40,7 +39,7 @@ Reducing the sample rate to a fraction of all transactions can make a huge diffe
4039
=== Transaction Queue
4140

4241
To reduce the load on the APM Server, the agent does not send every transaction up as it happens.
43-
Instead, it queues them up, and flushes the queue periodically, or when it reaches a maximum size, using a background thread.
42+
Instead, it queues them up and flushes the queue periodically, or when it reaches a maximum size, using a background thread.
4443

4544
While this reduces the load on the APM Server (and to a certain extent on the agent),
4645
holding on to the transaction data in a queue uses memory.
@@ -57,21 +56,20 @@ The second setting, `max_queue_size`, can help if you experience peaks of transa
5756
Keep in mind that reducing the value of either setting will cause the agent to send more HTTP requests to the APM Server,
5857
potentially causing a higher load.
5958

60-
6159
[float]
6260
[[tuning-max-spans]]
6361
=== Spans per transaction
6462

6563
The average amount of spans per transaction can influence how much time the agent spends in each transaction collecting contextual data for each span,
66-
and the the storage space needed in Elasticsearch.
67-
In our experience, most usual transactions should have well below 100 spans.
68-
In some cases however, the number of spans can explode:
64+
and the storage space needed in Elasticsearch.
65+
In our experience, most _usual_ transactions should have well below 100 spans.
66+
In some cases, however, the number of spans can explode:
6967

7068
* long-running transactions
7169
* unoptimized code, e.g. doing hundreds of SQL queries in a loop
7270

73-
To avoid that such edge cases overload both the agent and the APM Server,
74-
the agent stops recording spans when a limit is reached.
71+
To avoid these edge cases overloading both the agent and the APM Server,
72+
the agent stops recording spans when a specified limit is reached.
7573
You can configure this limit by changing the <<config-transaction-max-spans,`transaction_max_spans`>> setting.
7674

7775
Another option to reduce overhead of collecting contextual data for spans is to disable collection for very short spans.
@@ -81,3 +79,20 @@ You can define a minimal threshold for span duration in milliseconds,
8179
using the <<config-span-frames-min-duration,`span_frames_min_duration_ms`>> setting.
8280
If a span takes less than this duration, no stack frames will be collected for this span.
8381
Other contextual information, like the SQL query, will still be available.
82+
83+
[float]
84+
[[tuning-frame-context]]
85+
=== Collecting Frame Context
86+
87+
When a stack trace is captured, the agent will also capture several lines of source code around each frame location in the stack trace. This allows the APM UI to give greater insight into where exactly the error or span happens.
88+
89+
There are four settings you can modify to control this behavior:
90+
91+
* <<config-source-lines-error-app-frames, `source_lines_error_app_frames`>>
92+
* <<config-source-lines-error-library-frames,`source_lines_error_library_frames`>>
93+
* <<config-source-lines-span-app-frames,`source_lines_span_app_frames`>>
94+
* <<config-source-lines-span-library-frames,`source_lines_span_library_frames`>>
95+
96+
As you can see, these settings are divided between app frames, which represent your application code, and library frames, which represent the code of your dependencies. Each of these categories are also split into separate error and span settings.
97+
98+
Reading source files inside a running application can cause a lot of disk I/O, and sending up source lines for each frame will have a network and storage cost that is quite high. Turning down these limits will help prevent excessive memory usage.

0 commit comments

Comments
 (0)