Skip to content

Commit 2c3c905

Browse files
tsloughtertsloughter
authored
roll_dice_elli: update deps and improve readme description (#556)
* roll_dice_elli: bump dependencies * roll_dice_elli: add setup and traces section to readme * roll_dice_elli: add to readme details on setup, tracing and metrics
1 parent 0a8ec49 commit 2c3c905

File tree

8 files changed

+227
-70
lines changed

8 files changed

+227
-70
lines changed

‎examples/roll_dice_elli/README.md‎

Lines changed: 188 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,12 @@
11
roll_dice
22
=====
33

4+
Start the OpenTelemetry collector and Jaeger with Docker Compose:
5+
6+
```
7+
$ docker compose up
8+
```
9+
410
Run the project:
511

612
```
@@ -15,3 +21,185 @@ $ curl localhost:3000/rolldice
1521
```
1622

1723
Or go to `localhost:3000` in a browser to roll.
24+
25+
View traces in Jaeger at [http://localhost:16686](http://localhost:16686).d
26+
27+
# How It Works
28+
29+
## OpenTelemetry Setup
30+
31+
In `rebar.config` the dependency on `opentelemetry` and
32+
`opentelemetry_experimental` found in both `deps` as well as `shell` and `relx`
33+
ensures the OpenTelemetry SDK and Experimental SDK are part of the build and run
34+
of the project. The order of the lists of apps given to `shell` and `relx` are
35+
important here:
36+
37+
```erlang
38+
{shell, [{apps, [opentelemetry_exporter,
39+
opentelemetry_experimental,
40+
opentelemetry,
41+
recon,
42+
roll_dice]},
43+
{config, "config/sys.config"}]}.
44+
45+
{relx, [{release, {roll_dice, "0.1.0"},
46+
[opentelemetry_exporter,
47+
opentelemetry_experimental,
48+
opentelemetry,
49+
recon,
50+
roll_dice,
51+
sasl]}
52+
]}.
53+
```
54+
55+
These configurations tell Erlang to boot the exporter Application before the SDK
56+
both when running with `rebar3 shell` and when building an OTP Release and
57+
booting it. The exporter must start before the SDK to guarantee the modules are
58+
loaded before the SDK will call the exporter modules to initialize any OTLP
59+
exporter used.
60+
61+
The order also guarantees that on shutdown our application, `roll_dice`, is
62+
stopped before OpenTelemetry and that the exporter is shutdown only after the
63+
SDK has been.
64+
65+
The `applications` list in `roll_dice.app.src` should only need to specify the
66+
OpenTelemetry API since that is all it uses, which is expected for the end user
67+
application code.
68+
69+
## Traces
70+
71+
In `sys.config` the configuration for the OpenTelemetry SDK can be found:
72+
73+
```erlang
74+
{opentelemetry,
75+
[{span_processor, batch},
76+
{traces_exporter, otlp}
77+
]},
78+
```
79+
80+
This has the SDK setup to use the batch processor which batches up spans
81+
together before passing them on to the exporter, and to use the OTLP exporter.
82+
Defaults for the exporter are used since we just have it running in Docker on
83+
the default port and localhost.
84+
85+
Spans are created both by the `opentelemetry_elli` instrumentation library and
86+
by the handler module that handles each incoming HTTP request.
87+
`opentelemetry_elli` is integrated into the application as a middleware:
88+
89+
```erlang
90+
ElliOpts = [{callback, elli_middleware},
91+
{callback_args, [{mods, [{otel_elli_middleware, []},
92+
{roll_dice_handler, []}]}]},
93+
{port, Port}],
94+
95+
ChildSpecs = [#{id => roll_dice_http,
96+
start => {elli, start_link, [ElliOpts]},
97+
restart => permanent,
98+
shutdown => 5000,
99+
type => worker,
100+
modules => [roll_dice_handler]}],
101+
```
102+
103+
These options tell Elli to use `elli_middleware` for each request, this module
104+
will then call `otel_elli_middleware` followed by `roll_dice_handler`.
105+
106+
For each request the `elli_middleware` handler will first attempt to extract
107+
trace context from the request headers. It will then start a new span, with
108+
the span extracted from the trace context as the parent if it is available.
109+
110+
Because Elli has no router these spans need to have their name updated within
111+
`roll_dice_handler` to be more useful than simply being the HTTP method. Elli by
112+
default names spans only with the HTTP method name because the URL path is a
113+
high cardinality value submitted by the end user.
114+
115+
```erlang
116+
handle('GET', [<<"rolldice">>], _Req) ->
117+
?update_name(<<"GET /rolldice">>),
118+
...
119+
handle('GET', [], _Req) ->
120+
?update_name(<<"GET /">>),
121+
```
122+
123+
This handler code uses the `?update_name` macro to update the name of the
124+
current active span in the process dictionary context.
125+
126+
The core logic of the application is `do_roll`, shown here without any calls to
127+
metric related functions/macros:
128+
129+
```
130+
-spec do_roll() -> integer().
131+
do_roll() ->
132+
?with_span(dice_roll, #{},
133+
fun(_) ->
134+
Roll = rand:uniform(6),
135+
?set_attribute('roll.value', Roll),
136+
Roll
137+
end).
138+
```
139+
140+
The `?with_span` macro will start a new span named `dice_roll` with no
141+
attributes (`#{}`) and set it to be the active span within the process
142+
dictionary context while the anonymous function is run. When that function
143+
completes the span is ended. Within the body of that function a random number is
144+
generated and used as the attribute value for a span attribute `roll.value`.
145+
This attribute is added through the macro `?set_attribute` that will add an
146+
attribute to the currently active span in the process dictionary.
147+
148+
## Metrics
149+
150+
For metrics the Experimental SDK is configured in `sys.config`:
151+
152+
```erlang
153+
{opentelemetry_experimental,
154+
[{readers, [#{module => otel_metric_reader,
155+
config => #{export_interval_ms => 1000,
156+
exporter => {otel_metric_exporter_console, #{}}}}]}]},
157+
```
158+
159+
This configures the SDK to have a single metric reader, using module
160+
`otel_metric_reader`, that is configured to export metrics every second with an
161+
exporter that writes to stdout. This results in output to the console showing
162+
the metric name along with its attributes followed by the aggregate value, like:
163+
164+
```
165+
roll_counter{roll.value=5} 1
166+
```
167+
168+
Instruments have names which allow you to reference them from anywhere in your
169+
code you have that name. To help with this there is `roll_dice_instruments.hrl`:
170+
171+
```erlang
172+
-define(ROLL_COUNTER, roll_counter).
173+
```
174+
175+
To initialize the instruments used a call to `create_instruments/0` is added in
176+
`start/2` of `roll_dice_app`:
177+
178+
```erlang
179+
start(_StartType, _StartArgs) ->
180+
create_instruments(),
181+
roll_dice_sup:start_link().
182+
183+
create_instruments() ->
184+
?create_counter(?ROLL_COUNTER, #{description => <<"The number of rolls by roll value.">>,
185+
unit => '1'}).
186+
```
187+
188+
This means that when the `roll_dice` Application boots it will first create an
189+
instrument using the `?create_counter` macro and name it `?ROLL_COUNTER`.
190+
191+
In the handler, `roll_dice_handler`, the header `roll_dice_instruments.hrl` is
192+
included so the counter can be incremented with the result of the roll as an
193+
attribute on the measurement:
194+
195+
```erlang
196+
-spec do_roll() -> integer().
197+
do_roll() ->
198+
?with_span(dice_roll, #{},
199+
fun(_) ->
200+
Roll = rand:uniform(6),
201+
?set_attribute('roll.value', Roll),
202+
?counter_add(?ROLL_COUNTER, 1, #{'roll.value' => Roll}),
203+
Roll
204+
end).
205+
```

‎examples/roll_dice_elli/config/sys.config‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,10 +3,15 @@
33

44
{opentelemetry_elli, [{excluded_paths, ["/static", "/favicon.ico"]}]},
55

6+
{opentelemetry,
7+
[{span_processor, batch},
8+
{traces_exporter, otlp}
9+
]},
10+
611
{opentelemetry_experimental,
712
[{readers, [#{module => otel_metric_reader,
813
config => #{export_interval_ms => 1000,
9-
exporter => {opentelemetry_exporter, #{}}}}]}]},
14+
exporter => {otel_metric_exporter_console, #{}}}}]}]},
1015

1116
{kernel,
1217
[{logger_level, debug},

‎examples/roll_dice_elli/docker-compose.yml‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
1-
version: "3"
21
services:
32
otel:
4-
image: otel/opentelemetry-collector-contrib:0.76.1
3+
image: otel/opentelemetry-collector-contrib:0.134.1
54
command: ["--config=/conf/otel-collector-config.yaml"]
65
privileged: true
76
ports:
@@ -14,7 +13,7 @@ services:
1413
- jaeger-all-in-one
1514

1615
jaeger-all-in-one:
17-
image: jaegertracing/all-in-one:1.45
16+
image: jaegertracing/all-in-one:1.64.0
1817
restart: always
1918
environment:
2019
COLLECTOR_OTLP_ENABLED: true

‎examples/roll_dice_elli/otel-collector-config.yaml‎

Lines changed: 0 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -9,25 +9,18 @@ receivers:
99
allowed_origins:
1010
- "http://localhost:3000"
1111
endpoint: "0.0.0.0:4318"
12-
processors:
1312
exporters:
1413
otlp:
1514
endpoint: jaeger-all-in-one:4317
1615
tls:
1716
insecure: true
1817
sending_queue:
1918
batch:
20-
2119
debug:
2220
verbosity: detailed
2321
sampling_initial: 1
2422
sampling_thereafter: 1
25-
2623
service:
27-
telemetry:
28-
logs:
29-
level: "debug"
30-
extensions: [zpages]
3124
pipelines:
3225
traces:
3326
receivers: [otlp]

‎examples/roll_dice_elli/rebar.config‎

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -6,17 +6,20 @@
66
opentelemetry,
77
opentelemetry_elli,
88

9-
{opentelemetry_api_experimental,
10-
{git_subdir, "http://github.com/tsloughter/opentelemetry-erlang",
11-
{branch, "main"}, "apps/opentelemetry_api_experimental"}},
12-
{opentelemetry_experimental,
13-
{git_subdir, "http://github.com/tsloughter/opentelemetry-erlang",
14-
{branch, "main"}, "apps/opentelemetry_experimental"}}
9+
{opentelemetry_api_experimental, "~> 0.5"},
10+
{opentelemetry_experimental, "~> 0.5"}
1511
]}.
1612

1713
%% builds js/app.js into priv/static/index.js
1814
{pre_hooks, [{compile, "npm run build"}]}.
1915

16+
{shell, [{apps, [opentelemetry_exporter,
17+
opentelemetry_experimental,
18+
opentelemetry,
19+
recon,
20+
roll_dice]},
21+
{config, "config/sys.config"}]}.
22+
2023
{relx, [{release, {roll_dice, "0.1.0"},
2124
[opentelemetry_exporter,
2225
opentelemetry_experimental,
@@ -25,4 +28,3 @@
2528
roll_dice,
2629
sasl]}
2730
]}.
28-

‎examples/roll_dice_elli/rebar.lock‎

Lines changed: 21 additions & 51 deletions
Original file line numberDiff line numberDiff line change
@@ -1,70 +1,40 @@
11
{"1.2.0",
22
[{<<"acceptor_pool">>,{pkg,<<"acceptor_pool">>,<<"1.0.0">>},2},
3-
{<<"chatterbox">>,{pkg,<<"ts_chatterbox">>,<<"0.13.0">>},2},
3+
{<<"chatterbox">>,{pkg,<<"ts_chatterbox">>,<<"0.15.1">>},2},
44
{<<"ctx">>,{pkg,<<"ctx">>,<<"0.6.0">>},2},
55
{<<"elli">>,{pkg,<<"elli">>,<<"3.3.0">>},0},
6-
{<<"gproc">>,{pkg,<<"gproc">>,<<"0.8.0">>},2},
7-
{<<"grpcbox">>,{pkg,<<"grpcbox">>,<<"0.16.0">>},1},
8-
{<<"hpack">>,{pkg,<<"hpack_erl">>,<<"0.2.3">>},3},
9-
{<<"opentelemetry">>,
10-
{git_subdir,"http://github.com/tsloughter/opentelemetry-erlang",
11-
{ref,"1fc3bc54d9954048014df7fd6227c6b783fd7047"},
12-
"apps/opentelemetry"},
13-
0},
14-
{<<"opentelemetry_api">>,
15-
{git_subdir,"http://github.com/tsloughter/opentelemetry-erlang",
16-
{ref,"1fc3bc54d9954048014df7fd6227c6b783fd7047"},
17-
"apps/opentelemetry_api"},
18-
0},
19-
{<<"opentelemetry_api_experimental">>,
20-
{git_subdir,"http://github.com/tsloughter/opentelemetry-erlang",
21-
{ref,"1fc3bc54d9954048014df7fd6227c6b783fd7047"},
22-
"apps/opentelemetry_api_experimental"},
23-
0},
6+
{<<"gproc">>,{pkg,<<"gproc">>,<<"0.9.1">>},2},
7+
{<<"grpcbox">>,{pkg,<<"grpcbox">>,<<"0.17.1">>},1},
8+
{<<"hpack">>,{pkg,<<"hpack_erl">>,<<"0.3.0">>},3},
249
{<<"opentelemetry_elli">>,{pkg,<<"opentelemetry_elli">>,<<"0.2.0">>},0},
25-
{<<"opentelemetry_experimental">>,
26-
{git_subdir,"http://github.com/tsloughter/opentelemetry-erlang",
27-
{ref,"1fc3bc54d9954048014df7fd6227c6b783fd7047"},
28-
"apps/opentelemetry_experimental"},
29-
0},
30-
{<<"opentelemetry_exporter">>,
31-
{pkg,<<"opentelemetry_exporter">>,<<"1.4.1">>},
32-
0},
33-
{<<"opentelemetry_semantic_conventions">>,
34-
{pkg,<<"opentelemetry_semantic_conventions">>,<<"0.2.0">>},
35-
1},
36-
{<<"recon">>,{pkg,<<"recon">>,<<"2.5.3">>},0},
37-
{<<"ssl_verify_fun">>,{pkg,<<"ssl_verify_fun">>,<<"1.1.6">>},2},
10+
{<<"recon">>,{pkg,<<"recon">>,<<"2.5.6">>},0},
11+
{<<"ssl_verify_fun">>,{pkg,<<"ssl_verify_fun">>,<<"1.1.7">>},2},
3812
{<<"tls_certificate_check">>,
39-
{pkg,<<"tls_certificate_check">>,<<"1.18.0">>},
13+
{pkg,<<"tls_certificate_check">>,<<"1.23.0">>},
4014
1}]}.
4115
[
4216
{pkg_hash,[
4317
{<<"acceptor_pool">>, <<"43C20D2ACAE35F0C2BCD64F9D2BDE267E459F0F3FD23DAB26485BF518C281B21">>},
44-
{<<"chatterbox">>, <<"6F059D97BCAA758B8EA6FFFE2B3B81362BD06B639D3EA2BB088335511D691EBF">>},
18+
{<<"chatterbox">>, <<"5CAC4D15DD7AD61FC3C4415CE4826FC563D4643DEE897A558EC4EA0B1C835C9C">>},
4519
{<<"ctx">>, <<"8FF88B70E6400C4DF90142E7F130625B82086077A45364A78D208ED3ED53C7FE">>},
4620
{<<"elli">>, <<"089218762A7FF3D20AE81C8E911BD0F73EE4EE0ED85454226D1FC6B4FFF3B4F6">>},
47-
{<<"gproc">>, <<"CEA02C578589C61E5341FCE149EA36CCEF236CC2ECAC8691FBA408E7EA77EC2F">>},
48-
{<<"grpcbox">>, <<"B83F37C62D6EECA347B77F9B1EC7E9F62231690CDFEB3A31BE07CD4002BA9C82">>},
49-
{<<"hpack">>, <<"17670F83FF984AE6CD74B1C456EDDE906D27FF013740EE4D9EFAA4F1BF999633">>},
21+
{<<"gproc">>, <<"F1DF0364423539CF0B80E8201C8B1839E229E5F9B3CCB944C5834626998F5B8C">>},
22+
{<<"grpcbox">>, <<"6E040AB3EF16FE699FFB513B0EF8E2E896DA7B18931A1EF817143037C454BCCE">>},
23+
{<<"hpack">>, <<"2461899CC4AB6A0EF8E970C1661C5FC6A52D3C25580BC6DD204F84CE94669926">>},
5024
{<<"opentelemetry_elli">>, <<"0FECED66B5D3C4590C249505339FBA8FFE1B3C2CB939B2EABD3574A690CC1F01">>},
51-
{<<"opentelemetry_exporter">>, <<"5C80C3A22EC084B4E0A9AC7D39A435B332949B2DCEEC9FB19F5C5D2CA8AE1D56">>},
52-
{<<"opentelemetry_semantic_conventions">>, <<"B67FE459C2938FCAB341CB0951C44860C62347C005ACE1B50F8402576F241435">>},
53-
{<<"recon">>, <<"739107B9050EA683C30E96DE050BC59248FD27EC147696F79A8797FF9FA17153">>},
54-
{<<"ssl_verify_fun">>, <<"CF344F5692C82D2CD7554F5EC8FD961548D4FD09E7D22F5B62482E5AEAEBD4B0">>},
55-
{<<"tls_certificate_check">>, <<"75699BC855EA18DE358E3024ABD73384691320BB7A0C98AC90A74475311C1AE3">>}]},
25+
{<<"recon">>, <<"9052588E83BFEDFD9B72E1034532AEE2A5369D9D9343B61AEB7FBCE761010741">>},
26+
{<<"ssl_verify_fun">>, <<"354C321CF377240C7B8716899E182CE4890C5938111A1296ADD3EC74CF1715DF">>},
27+
{<<"tls_certificate_check">>, <<"BB7869C629DE4EC72D4652520C1AD2255BB5712AD09A6568C41B0294B3CEC78F">>}]},
5628
{pkg_hash_ext,[
5729
{<<"acceptor_pool">>, <<"0CBCD83FDC8B9AD2EEE2067EF8B91A14858A5883CB7CD800E6FCD5803E158788">>},
58-
{<<"chatterbox">>, <<"B93D19104D86AF0B3F2566C4CBA2A57D2E06D103728246BA1AC6C3C0FF010AA7">>},
30+
{<<"chatterbox">>, <<"4F75B91451338BC0DA5F52F3480FA6EF6E3A2AEECFC33686D6B3D0A0948F31AA">>},
5931
{<<"ctx">>, <<"A14ED2D1B67723DBEBBE423B28D7615EB0BDCBA6FF28F2D1F1B0A7E1D4AA5FC2">>},
6032
{<<"elli">>, <<"698B13B33D05661DB9FE7EFCBA41B84825A379CCE86E486CF6AFF9285BE0CCF8">>},
61-
{<<"gproc">>, <<"580ADAFA56463B75263EF5A5DF4C86AF321F68694E7786CB057FD805D1E2A7DE">>},
62-
{<<"grpcbox">>, <<"294DF743AE20A7E030889F00644001370A4F7CE0121F3BBDAF13CF3169C62913">>},
63-
{<<"hpack">>, <<"06F580167C4B8B8A6429040DF36CC93BBA6D571FAEAEC1B28816523379CBB23A">>},
33+
{<<"gproc">>, <<"905088E32E72127ED9466F0BAC0D8E65704CA5E73EE5A62CB073C3117916D507">>},
34+
{<<"grpcbox">>, <<"4A3B5D7111DAABC569DC9CBD9B202A3237D81C80BF97212FBC676832CB0CEB17">>},
35+
{<<"hpack">>, <<"D6137D7079169D8C485C6962DFE261AF5B9EF60FBC557344511C1E65E3D95FB0">>},
6436
{<<"opentelemetry_elli">>, <<"41A8DBDA4781179357E7C6EF40BEC8E9DB9E7637A63C2AAE8B740F099ADD4392">>},
65-
{<<"opentelemetry_exporter">>, <<"5A0FF6618B0F7370BD10B50E64099A4C2AA52145AE6567CCCF7D76BA2D32E079">>},
66-
{<<"opentelemetry_semantic_conventions">>, <<"D61FA1F5639EE8668D74B527E6806E0503EFC55A42DB7B5F39939D84C07D6895">>},
67-
{<<"recon">>, <<"6C6683F46FD4A1DFD98404B9F78DCABC7FCD8826613A89DCB984727A8C3099D7">>},
68-
{<<"ssl_verify_fun">>, <<"BDB0D2471F453C88FF3908E7686F86F9BE327D065CC1EC16FA4540197EA04680">>},
69-
{<<"tls_certificate_check">>, <<"8DE62FF34E59317211567E3B1AE70866DF195895B051193C21ABC381276D395B">>}]}
37+
{<<"recon">>, <<"96C6799792D735CC0F0FD0F86267E9D351E63339CBE03DF9D162010CEFC26BB0">>},
38+
{<<"ssl_verify_fun">>, <<"FE4C190E8F37401D30167C8C405EDA19469F34577987C76DDE613E838BBC67F8">>},
39+
{<<"tls_certificate_check">>, <<"79D0C84EFFC7C81AC1E85FA38B1C33572FE2976FB8FAFDFB2F0140DE0442D494">>}]}
7040
].

‎examples/roll_dice_elli/src/roll_dice.app.src‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
[kernel,
88
stdlib,
99
elli,
10+
tls_certificate_check,
1011
opentelemetry_api,
1112
opentelemetry_api_experimental,
1213
opentelemetry_elli

‎examples/roll_dice_elli/src/roll_dice_sup.erl‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,6 @@ init([]) ->
2828
{roll_dice_handler, []}]}]},
2929
{port, Port}],
3030

31-
3231
ChildSpecs = [#{id => roll_dice_http,
3332
start => {elli, start_link, [ElliOpts]},
3433
restart => permanent,

0 commit comments

Comments
 (0)