Skip to content

Commit 636f305

Browse files
nerda-codesnerda-codes
authored
docs(audit-trail): doc review
1 parent 77f189f commit 636f305

File tree

1 file changed

+157
-150
lines changed
  • tutorials/export-audit-trail-to-datadog

1 file changed

+157
-150
lines changed

‎tutorials/export-audit-trail-to-datadog/index.mdx‎

Lines changed: 157 additions & 150 deletions
Original file line numberDiff line numberDiff line change
@@ -1,167 +1,174 @@
11
---
22
meta:
3-
title: Export Audit Trail to DataDog
3+
title: Exporting Audit Trail events to DataDog
44
description: Learn how to export audit trail events to DataDog
55
content:
6-
h1: Export Audit Trail to DataDog
6+
h1: Exporting Audit Trail events to DataDog
77
paragraph: Learn how to export audit trail events to DataDog
88
tags: audit-trail log events
99
categories:
1010
- audit-trail
1111
- instances
1212
dates:
13-
validation: 2025-02-06
14-
posted: 2025-02-06
13+
validation: 2025-02-10
14+
posted: 2025-02-10
1515
---
1616

17-
This guide will help you exporting audit trail events to DataDog. For that, it will depends on building a [custom OpenTelemetry Collector](https://opentelemetry.io/docs/collector/custom-collector/) that will collect Audit Trail events through the [Audit Trail receiver](https://github.com/scaleway/opentelemetry-collector-scaleway/tree/main/receiver/scwaudittrail) and export them with the [DataDog exporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/datadogexporter).
17+
This tutorial shows you how to export your Audit Trail events to [DataDog](https://www.datadoghq.com/). For the purpose of this tutorail, we are building a [custom OpenTelemetry Collector](https://opentelemetry.io/docs/collector/custom-collector/) to collect Audit Trail events through the [Audit Trail receiver](https://github.com/scaleway/opentelemetry-collector-scaleway/tree/main/receiver/scwaudittrail) and export them with the [DataDog exporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/datadogexporter).
1818

1919
<Macro id="requirements" />
2020

2121
- A Scaleway account logged into the [console](https://console.scaleway.com)
2222
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
23-
- An [SSH key](/organizations-and-projects/how-to/create-ssh-key/)
24-
- An [Instance](/instances/how-to/create-an-instance/)
25-
26-
## Building the collector
27-
28-
The first step is to install the OpenTelemetry Collector Builder by following [this link](https://opentelemetry.io/docs/collector/custom-collector/#step-1---install-the-builder).
29-
30-
Once you have the `ocb` binary, you will create the manifest in YAML to configure the builder. Create a file `builder-config.yaml` with the following content:
31-
32-
```yaml
33-
dist:
34-
name: otelcol-audit-trail
35-
description: OpenTelemetry Collector for Audit Trail
36-
output_path: ./otelcol-audit-trail
37-
38-
exporters:
39-
- gomod:
40-
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/datadogexporter v0.118.0
41-
42-
processors:
43-
- gomod:
44-
go.opentelemetry.io/collector/processor/batchprocessor v0.118.0
45-
46-
receivers:
47-
- gomod:
48-
github.com/scaleway/opentelemetry-collector-scaleway/receiver/scwaudittrail v0.1.0
49-
50-
providers:
51-
- gomod: go.opentelemetry.io/collector/confmap/provider/envprovider v1.24.0
52-
- gomod: go.opentelemetry.io/collector/confmap/provider/fileprovider v1.24.0
53-
- gomod: go.opentelemetry.io/collector/confmap/provider/httpprovider v1.24.0
54-
- gomod: go.opentelemetry.io/collector/confmap/provider/httpsprovider v1.24.0
55-
- gomod: go.opentelemetry.io/collector/confmap/provider/yamlprovider v1.24.0
56-
```
57-
58-
Then you can build the collector by running the following command. We specify `GOOS` and `GOARCH` because the target deployment is linux with an AMD64 CPU.
59-
60-
```
61-
GOOS=linux GOARCH=amd64 ./ocb --config builder-config.yaml
62-
```
63-
64-
You will have a new folder named `otelcol-audit-trail/` with the binary compiled inside named `otelcol-audit-trail`.
65-
66-
## Deploying the collector
67-
68-
The next thing to do is to upload the collector binary to your instance:
69-
70-
```
71-
scp otelcol-audit-trail/otelcol-audit-trail root@<IP ADDRESS>:/usr/local/bin/
72-
```
73-
74-
The remaining of the tutoial will happen inside the instance, you need to ssh to it.
75-
76-
```
77-
ssh root@<IP ADDRESS>
78-
```
79-
80-
## Configure the collector
81-
82-
The custom collector we just build needs a configuration to run. Create the file `/etc/opentelemetry-collector/config.yaml` with the following content:
83-
84-
```yaml
85-
receivers:
86-
scwaudittrail:
87-
access_key: <SCW_ACCESS_KEY>
88-
secret_key: <SCW_SECRET_KEY>
89-
organization_id: <SCW_DEFAULT_ORGANIZATION_ID>
90-
region: <SCW_DEFAULT_REGION>
91-
92-
processors:
93-
batch:
94-
send_batch_max_size: 1000
95-
send_batch_size: 100
96-
timeout: 10s
97-
98-
exporters:
99-
datadog:
100-
idle_conn_timeout: 10s
101-
api:
102-
key: <DD_API_KEY>
103-
site: <DD_SITE>
104-
105-
service:
106-
pipelines:
107-
logs:
108-
receivers: [scwaudittrail]
109-
processors: [batch]
110-
exporters: [datadog]
111-
```
112-
113-
Be sure to replace the following variables:
114-
- SCW_ACCESS_KEY: Scaleway API access key
115-
- SCW_SECRET_KEY: Scaleway API secret key
116-
- SCW_DEFAULT_ORGANIZATION_ID: Scaleway organization ID
117-
- SCW_DEFAULT_REGION: Scaleway region
118-
- DD_API_KEY: DataDog API key
119-
- DD_SITE: DataDog site (see documentation in [DataDog site](https://docs.datadoghq.com/getting_started/site/#access-the-datadog-site))
120-
121-
## Running the collector
122-
123-
Create the systemd service that will run the collector by creating the file `/etc/systemd/system/opentelemetry-collector.service` with the following content:
124-
125-
```
126-
[Unit]
127-
Description=OpenTelemetry Collector
128-
After=multi-user.target
129-
130-
[Service]
131-
ExecStart=/usr/local/bin/otelcol-audit-trail --config /etc/opentelemetry-collector/config.yaml
132-
Type=simple
133-
134-
[Install]
135-
WantedBy=multi-user.target
136-
```
137-
138-
Run the following command to update systemd services:
139-
140-
```
141-
systemctl daemon-reload
142-
```
143-
144-
Then you can enable and start the service by running:
145-
146-
```
147-
systemctl enable opentelemetry-collector.service
148-
systemctl start opentelemetry-collector.service
149-
```
150-
151-
You can ensure the service is running with the command:
152-
153-
```
154-
systemctl status opentelemetry-collector.service
155-
```
156-
157-
And you can follow the logs with the command
158-
159-
```
160-
journalctl -fu opentelemetry-collector.service
161-
```
162-
163-
To confirm that the collector is polling Audit Trail events you should see something like this in the logs:
164-
165-
```
166-
Feb 07 15:34:30 scw-beautiful-zhukovsky otelcol-audit-trail[1723]: 2025-02-07T15:34:30.687Z info [email protected]/receiver.go:80 Polling Audit Trail logs {"kind": "receiver", "name": "scwaudittrail", "data_type": "logs"}
167-
```
23+
- Created an [SSH key](/organizations-and-projects/how-to/create-ssh-key/)
24+
- Created an [Instance](/instances/how-to/create-an-instance/)
25+
- Installed the [OpenTelemetry collector builder](https://opentelemetry.io/docs/collector/custom-collector/#step-1---install-the-builder)
26+
- Created a [Datadog account](https://app.datadoghq.com/account/login) and a [Datadog API key](https://docs.datadoghq.com/account_management/api-app-keys/#api-keys)
27+
28+
## Building the OpenTelemetry collector
29+
30+
1. Open a terminal and check that the `ocb` binary is ready to be used. The output of the `help` command should display, meaning the `ocb` binary is ready to be used.
31+
32+
```
33+
./ocb help
34+
```
35+
36+
2. Create a manifest file named `builder-config.yaml` and paste the following content into it. This file is used to defines code generation, the compile process, and the components to include in your Collector’s distribution.
37+
38+
```yaml
39+
dist:
40+
name: otelcol-audit-trail
41+
description: OpenTelemetry Collector for Audit Trail
42+
output_path: ./otelcol-audit-trail
43+
44+
exporters:
45+
- gomod:
46+
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/datadogexporter v0.118.0
47+
48+
processors:
49+
- gomod:
50+
go.opentelemetry.io/collector/processor/batchprocessor v0.118.0
51+
52+
receivers:
53+
- gomod:
54+
github.com/scaleway/opentelemetry-collector-scaleway/receiver/scwaudittrail v0.1.0
55+
56+
providers:
57+
- gomod: go.opentelemetry.io/collector/confmap/provider/envprovider v1.24.0
58+
- gomod: go.opentelemetry.io/collector/confmap/provider/fileprovider v1.24.0
59+
- gomod: go.opentelemetry.io/collector/confmap/provider/httpprovider v1.24.0
60+
- gomod: go.opentelemetry.io/collector/confmap/provider/httpsprovider v1.24.0
61+
- gomod: go.opentelemetry.io/collector/confmap/provider/yamlprovider v1.24.0
62+
```
63+
64+
3. Run the following command to build the Collector. `GOOS` and `GOARCH` are needed in the command as the target deployment is Linux with an AMD64 CPU.
65+
66+
```
67+
GOOS=linux GOARCH=amd64 ./ocb --config builder-config.yaml
68+
```
69+
70+
You now have a new folder named `otelcol-audit-trail/` with the binary `otelcol-audit-trail` compiled inside.
71+
72+
## Deploying the Collector
73+
74+
1. Run the following command to upload the Collector binary to your Instance. Make sure that you replace `<INSTANCE_IP_ADDRESS>` with the IP address of your Instance.
75+
76+
```
77+
scp otelcol-audit-trail/otelcol-audit-trail root@<INSTANCE_IP_ADDRESS>:/usr/local/bin/
78+
```
79+
80+
2. Connect to your Instance via SSH:
81+
82+
```
83+
ssh root@<INSTANCE_IP_ADDRESS>
84+
```
85+
86+
## Configure the Collector
87+
88+
Create a file named `/etc/opentelemetry-collector/config.yaml` and paste the following content into it. This file is the configuration our custom Collector will run.
89+
90+
```yaml
91+
receivers:
92+
scwaudittrail:
93+
access_key: <SCW_ACCESS_KEY>
94+
secret_key: <SCW_SECRET_KEY>
95+
organization_id: <SCW_DEFAULT_ORGANIZATION_ID>
96+
region: <SCW_DEFAULT_REGION>
97+
98+
processors:
99+
batch:
100+
send_batch_max_size: 1000
101+
send_batch_size: 100
102+
timeout: 10s
103+
104+
exporters:
105+
datadog:
106+
idle_conn_timeout: 10s
107+
api:
108+
key: <DD_API_KEY>
109+
site: <DD_SITE>
110+
111+
service:
112+
pipelines:
113+
logs:
114+
receivers: [scwaudittrail]
115+
processors: [batch]
116+
exporters: [datadog]
117+
```
118+
119+
Make sure that you replace:
120+
121+
- `SCW_ACCESS_KEY` with your Scaleway API access key
122+
- `SCW_SECRET_KEY` with your Scaleway API secret key
123+
- `SCW_DEFAULT_ORGANIZATION_ID` with your Scaleway Organization ID
124+
- `SCW_DEFAULT_REGION` with the Scaleway region to target
125+
- `DD_API_KEY` with your DataDog API key
126+
- `DD_SITE` with the [DataDog site](https://docs.datadoghq.com/getting_started/site/#access-the-datadog-site) you are on
127+
128+
## Running the Collector
129+
130+
1. Create a file named `/etc/systemd/system/opentelemetry-collector.service` and paste the following content into it. This file will create the `systemd` service that runs the Collector.
131+
132+
```
133+
[Unit]
134+
Description=OpenTelemetry Collector
135+
After=multi-user.target
136+
137+
[Service]
138+
ExecStart=/usr/local/bin/otelcol-audit-trail --config /etc/opentelemetry-collector/config.yaml
139+
Type=simple
140+
141+
[Install]
142+
WantedBy=multi-user.target
143+
```
144+
145+
2. Run the following command to update `systemd` services:
146+
147+
```
148+
systemctl daemon-reload
149+
```
150+
151+
3. Run the following command to enable and start the service:
152+
153+
```
154+
systemctl enable opentelemetry-collector.service
155+
systemctl start opentelemetry-collector.service
156+
```
157+
158+
4. Make sure that the service is running:
159+
160+
```
161+
systemctl status opentelemetry-collector.service
162+
```
163+
164+
5. Run the command below to visualize your logs:
165+
166+
```
167+
journalctl -fu opentelemetry-collector.service
168+
```
169+
170+
AN output similar to the following should display to confirm that the Collector is pulling Audit Trail events:
171+
172+
```
173+
Feb 07 15:34:30 scw-beautiful-zhukovsky otelcol-audit-trail[1723]: 2025-02-07T15:34:30.687Z info [email protected]/receiver.go:80 Polling Audit Trail logs {"kind": "receiver", "name": "scwaudittrail", "data_type": "logs"}
174+
```

0 commit comments

Comments
 (0)