|
| 1 | +# Quick start on exporting metrics to Prometheus/Grafana |
| 2 | + |
| 3 | +- [Quick start on exporting metrics to Prometheus/Grafana](#quick-start-on-exporting-metrics-to-prometheusgrafana) |
| 4 | + - [Prerequisite](#prerequisite) |
| 5 | + - [Introduction](#introduction) |
| 6 | + - [Configure OpenTelemetry to Expose metrics via Prometheus Endpoint](#configure-opentelemetry-to-expose-metrics-via-prometheus-endpoint) |
| 7 | + - [Check Results in the browser](#check-results-in-the-browser) |
| 8 | + - [Download Prometheus](#download-prometheus) |
| 9 | + - [Prometheus and Grafana](#prometheus-and-grafana) |
| 10 | + - [Configuration](#configuration) |
| 11 | + - [Start Prometheus](#start-prometheus) |
| 12 | + - [View Results in Prometheus](#view-results-in-prometheus) |
| 13 | + - [View/Query Results with Grafana](#viewquery-results-with-grafana) |
| 14 | + |
| 15 | +## Prerequisite |
| 16 | + |
| 17 | +It is highly recommended to go over the [getting-started](../getting-started/README.md) |
| 18 | +doc before following along this document. |
| 19 | + |
| 20 | +## Introduction |
| 21 | + |
| 22 | +- [What is Prometheus?](https://prometheus.io/docs/introduction/overview/) |
| 23 | + |
| 24 | +- [Grafana support for |
| 25 | + Prometheus](https://prometheus.io/docs/visualization/grafana/#creating-a-prometheus-graph) |
| 26 | + |
| 27 | +### Configure OpenTelemetry to Expose metrics via Prometheus Endpoint |
| 28 | + |
| 29 | +Create a new console application and run it: |
| 30 | + |
| 31 | +```sh |
| 32 | +dotnet new console --output prometheus-http-server |
| 33 | +cd prometheus-http-server |
| 34 | +dotnet run |
| 35 | +``` |
| 36 | + |
| 37 | +Add a reference to [prometheus |
| 38 | +exporter](https://www.nuget.org/packages/opentelemetry.exporter.prometheus) to |
| 39 | +this application. |
| 40 | + |
| 41 | +```shell |
| 42 | +dotnet add package OpenTelemetry.Exporter.Prometheus --version 1.2.0-rc1 |
| 43 | +``` |
| 44 | + |
| 45 | +Now, we are going to make some small tweaks to the example in the |
| 46 | +getting-started metrics `Program.cs` to make the metrics available via |
| 47 | +OpenTelemetry Prometheus Exporter. |
| 48 | + |
| 49 | +First, copy and paste everything from getting-started |
| 50 | +metrics [example](../getting-started/Program.cs) to the Program.cs file of the |
| 51 | +new console application (prometheus-http-server) we've created. |
| 52 | + |
| 53 | +And replace the below line: |
| 54 | + |
| 55 | +```csharp |
| 56 | +.AddConsoleExporter() |
| 57 | +``` |
| 58 | + |
| 59 | +with |
| 60 | + |
| 61 | +```csharp |
| 62 | +.AddPrometheusExporter(opt => |
| 63 | +{ |
| 64 | + opt.StartHttpListener = true; |
| 65 | + opt.HttpListenerPrefixes = new string[] { $"http://localhost:9184/" }; |
| 66 | +}) |
| 67 | +``` |
| 68 | + |
| 69 | +With `.AddPrometheusExporter()` function, OpenTelemetry `PrometheusExporter` will |
| 70 | +export data via the endpoint defined by `HttpListenerPrefixes`. |
| 71 | + |
| 72 | +Also, for our learning purpose, use a while-loop to keep increasing the counter |
| 73 | +value until any key is pressed. |
| 74 | + |
| 75 | +```csharp |
| 76 | +Console.WriteLine("Press any key to exit"); |
| 77 | +while (!Console.KeyAvailable) |
| 78 | +{ |
| 79 | + Thread.Sleep(1000); |
| 80 | + MyFruitCounter.Add(1, new("name", "apple"), new("color", "red")); |
| 81 | + MyFruitCounter.Add(2, new("name", "lemon"), new("color", "yellow")); |
| 82 | + MyFruitCounter.Add(1, new("name", "lemon"), new("color", "yellow")); |
| 83 | + ... |
| 84 | + ... |
| 85 | + ... |
| 86 | +} |
| 87 | +``` |
| 88 | + |
| 89 | +After the above modifications, now our `Program.cs` should look like [this](./Program.cs). |
| 90 | + |
| 91 | +### Check Results in the browser |
| 92 | + |
| 93 | +Start the application and leave the process running. Now we should be able to |
| 94 | +see the metrics at the endpoint we've defined in `Program.cs`; in this case, the |
| 95 | +endpoint is: "http://localhost:9184/". |
| 96 | + |
| 97 | +Check the output metrics with your favorite browser: |
| 98 | + |
| 99 | + |
| 100 | + |
| 101 | +Now, we understand how we can configure Opentelemetry `PrometheusExporter` to |
| 102 | +export metrics the endpoint we specified. Next, we are going to learn about how |
| 103 | +to use Prometheus and Grafana to view/query the metrics |
| 104 | +visualization. |
| 105 | + |
| 106 | +## Download Prometheus |
| 107 | + |
| 108 | +Follow the [first steps]((https://prometheus.io/docs/introduction/first_steps/)) |
| 109 | +to download the [latest release](https://prometheus.io/download/) of Prometheus. |
| 110 | + |
| 111 | +## Prometheus and Grafana |
| 112 | + |
| 113 | +### Configuration |
| 114 | + |
| 115 | +After finished downloading, extract it to a local location that's easy to |
| 116 | +access. We will find the default Prometheus configuration YAML file in the |
| 117 | +folder, named `prometheus.yml`. |
| 118 | + |
| 119 | +Let's create a new file in the same location as where `prometheus.yml` locates, |
| 120 | +and named the new file as `otel.yml` for this exercise. Then, copy and paste the |
| 121 | +entire content below into the otel.yml file we have created just now. |
| 122 | + |
| 123 | +```yaml |
| 124 | +global: |
| 125 | + scrape_interval: 10s |
| 126 | + scrape_timeout: 10s |
| 127 | + evaluation_interval: 10s |
| 128 | +scrape_configs: |
| 129 | +- job_name: MyOpenTelemetryDemo |
| 130 | + honor_timestamps: true |
| 131 | + scrape_interval: 1s |
| 132 | + scrape_timeout: 1s |
| 133 | + metrics_path: /metrics |
| 134 | + scheme: http |
| 135 | + follow_redirects: true |
| 136 | + static_configs: |
| 137 | + - targets: |
| 138 | + # set the target to the location where metrics will be exposed by |
| 139 | + # the OpenTelemetry Prometheus Exporter |
| 140 | + - localhost:9184 |
| 141 | +``` |
| 142 | +
|
| 143 | +### Start Prometheus |
| 144 | +
|
| 145 | +Follow the instructions from |
| 146 | +[starting-prometheus](https://prometheus.io/docs/introduction/first_steps/#starting-prometheus) |
| 147 | +to start the Prometheus server and verify it has been started successfully. |
| 148 | +
|
| 149 | +Please note that we will need pass in otel.yml file as the argument: |
| 150 | +
|
| 151 | +```console |
| 152 | +./prometheus --config.file=otel.yml |
| 153 | +``` |
| 154 | + |
| 155 | +### View Results in Prometheus |
| 156 | + |
| 157 | +To use the graphical interface for viewing our metrics with Prometheus, navigate |
| 158 | +to "http://localhost:9090/graph", and type `MyFruitCounter` in the expression |
| 159 | +bar of the UI; finally, click the execute button. |
| 160 | + |
| 161 | +We should be able to see the following chart from the browser: |
| 162 | + |
| 163 | + |
| 164 | + |
| 165 | +From the legend, we can see that the `instance` name and the `job` name are the |
| 166 | +values we have set in `otel.yml`. |
| 167 | + |
| 168 | +Congratulations! |
| 169 | + |
| 170 | +Now we know how to configure Prometheus server and deploy OpenTelemetry |
| 171 | +`PrometheusExporter` to export our metrics. Next, we are going to explore a tool |
| 172 | +called Grafana, which has powerful visualizations for the metrics. |
| 173 | + |
| 174 | +### View/Query Results with Grafana |
| 175 | + |
| 176 | +Please [Install Grafana](https://grafana.com/docs/grafana/latest/installation/). |
| 177 | + |
| 178 | +For windows users, after finishing installation, start the standalone Grafana |
| 179 | +server, grafana-server.exe located in the bin folder. Then, use the browser to |
| 180 | +navigate to the default port of Grafana `3000`. We can confirm the port number |
| 181 | +with the logs from the command line after starting the Grafana server as well. |
| 182 | + |
| 183 | +Follow the instructions in the Grafana getting started |
| 184 | +[doc](https://grafana.com/docs/grafana/latest/getting-started/getting-started/#step-2-log-in) |
| 185 | +to log in. |
| 186 | + |
| 187 | +After successfully logging in, click on the explore option on the left panel of |
| 188 | +the website - we should be able to write some queries to explore our metrics |
| 189 | +now! |
| 190 | + |
| 191 | +Feel free to find some handy PromQL |
| 192 | +[here](https://promlabs.com/promql-cheat-sheet/). |
| 193 | + |
| 194 | +In the below example, the query targets to find out what is the per-second rate |
| 195 | +of increace of myFruitCounter over the last 30 minutes: |
| 196 | + |
| 197 | + |
0 commit comments