Skip to content

Commit afabd52

Browse files
simranquirkysimranquirky
authored
Devops integrations (#111)
* added integration page for github actions * Added Integration page for Jenkins * added Integration page for Terraform * added Integration page for Ansible * added Integration page for nginx server --------- Co-authored-by: simranquirky <simranquirky>
1 parent 7be79ba commit afabd52

22 files changed

+720
-1
lines changed

‎docs/integration/.pages

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,4 +4,6 @@ nav:
44
- AWS: aws
55
- Cloudflare: cloudflare.md
66
- PostgreSQL: postgresql.md
7-
- Snowflake: snowflake.md
7+
- Snowflake: snowflake.md
8+
- Servers: servers
9+
- DevOps : devops

‎docs/integration/devops/.pages

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
nav:
2+
3+
- Jenkins: jenkins.md
4+
- Ansible: ansible.md
5+
- Terraform: terraform.md
6+
- Github Actions: github-actions.md
7+

‎docs/integration/devops/ansible.md

Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
---
2+
title: Ansible Logs Integration Guide
3+
description: Forward Ansible execution logs to OpenObserve using Fluent Bit for real-time search and observability.
4+
5+
---
6+
7+
# Integration with Ansible
8+
9+
This guide walks you through forwarding **Ansible logs** from an Ubuntu machine to OpenObserve using Fluent Bit.
10+
11+
## Overview
12+
13+
Ansible doesn’t send logs to external systems by default. This integration configures a local playbook to write logs to a file, and uses Fluent Bit to tail that file and send logs to OpenObserve over HTTP.
14+
15+
## Steps to Integrate
16+
17+
??? "Prerequisites"
18+
19+
- Ubuntu machine (local, virtual machine, or cloud VM)
20+
- User with `sudo` privileges
21+
- Basic familiarity with Ansible and Fluent Bit
22+
- OpenObserve account ([Cloud](https://cloud.openobserve.ai/web/) or [Self-Hosted](../../../quickstart/#self-hosted-installation))
23+
- Ansible Installed
24+
25+
??? "Step 1: Configure Ansible log File"
26+
27+
1. **Configure log output path in `ansible.cfg`**:
28+
```ini
29+
[defaults]
30+
log_path = /tmp/ansible_log_demo.txt
31+
```
32+
33+
??? "Step 2: Create a Local Ansible Task (Optional)"
34+
35+
1. **Create a sample playbook**:
36+
```bash
37+
vi local_task.yml
38+
```
39+
2. Add the following contents:
40+
```yaml
41+
---
42+
- name: Ansible Local Task Demo
43+
hosts: localhost
44+
connection: local
45+
tasks:
46+
- name: Create a sample file
47+
ansible.builtin.file:
48+
path: /tmp/ansible_log_demo.txt
49+
state: touch
50+
51+
- name: Write a message to the sample file
52+
ansible.builtin.copy:
53+
content: "This is a demo log entry from Ansible."
54+
dest: /tmp/ansible_log_demo.txt
55+
```
56+
57+
3. **Run the playbook**:
58+
```bash
59+
ansible-playbook local_task.yml
60+
```
61+
62+
After running, check `/tmp/ansible_log_demo.txt` to confirm logs are generated.
63+
64+
??? "Step 3: Install Fluent Bit on Ubuntu"
65+
66+
1. **Install Fluent Bit using the official script**:
67+
```bash
68+
curl https://raw.githubusercontent.com/fluent/fluent-bit/master/install.sh | sh
69+
```
70+
> For manual install or using package managers, refer to [Fluent Bit Docs](https://docs.fluentbit.io/manual/installation/linux/ubuntu)
71+
72+
2. **Verify installation**:
73+
```bash
74+
fluent-bit --version
75+
```
76+
77+
??? "Step 4: Configure Fluent Bit for Ansible Logs"
78+
79+
1. **Edit Fluent Bit config file**:
80+
```bash
81+
sudo vi /etc/fluent-bit/fluent-bit.conf
82+
```
83+
84+
2. **Add the following configuration**:
85+
```ini
86+
[SERVICE]
87+
Flush 5
88+
Daemon Off
89+
Log_Level info
90+
91+
[INPUT]
92+
Name tail
93+
Path /tmp/ansible_log_demo.txt
94+
Tag ansible.demo
95+
Refresh_Interval 5
96+
97+
[OUTPUT]
98+
Name http
99+
Match *
100+
URI /api/<O2_ORG_NAME>/<O2_STREAM_NAME>/_json
101+
Host <O2_HOST>
102+
Port 443
103+
tls On
104+
Format json
105+
Json_date_key _timestamp
106+
Json_date_format iso8601
107+
HTTP_User <O2_USER>
108+
HTTP_Passwd <O2_PASSWORD>
109+
compress gzip
110+
```
111+
112+
> Note
113+
- Replace `<O2_ORG_NAME>`, `<O2_STREAM_NAME>`, `<O2_HOST>`, `<O2_USER>`, and `<O2_PASSWORD>` with your OpenObserve values.
114+
- Example URI: `/api/default/ansible/_json` for the `default` org and `ansible` stream.
115+
116+
??? "Step 5: Start Fluent Bit"
117+
118+
1. **Start the Fluent Bit service**:
119+
```bash
120+
sudo systemctl start fluent-bit
121+
sudo systemctl enable fluent-bit
122+
```
123+
2. **Check service status**:
124+
```bash
125+
sudo systemctl status fluent-bit
126+
```
127+
> Ensure there are no startup errors. Logs will now be tailed and sent to OpenObserve.
128+
129+
??? "Step 6: Verify Logs in OpenObserve"
130+
131+
1. In your Openobserve instance, Go to **Logs** → select your log stream → Set time range → Click **Run Query**
132+
133+
![Verify Ansible Logs in OpenObserve](https://openobserve.ai/assets%2Fansible_logs_o2_2fa50f03c6.gif)
134+
## Troubleshooting
135+
136+
??? "No logs in OpenObserve?"
137+
138+
- Ensure Fluent Bit is running: `sudo systemctl status fluent-bit`
139+
- Double-check the HTTP config and authentication
140+
- Use Fluent Bit in debug mode:
141+
```bash
142+
sudo fluent-bit -c /etc/fluent-bit/fluent-bit.conf -vv
143+
```
144+
145+
??? "Ansible log file not updating?"
146+
147+
- Confirm `log_path` is correctly set in `ansible.cfg`
148+
- Use verbose flags (`-v`, `-vv`, `-vvv`) to produce more logs
149+
- Confirm Ansible has permission to write to `/tmp`
150+
151+
??? "Fluent Bit not picking up logs?"
152+
153+
- Make sure the path `/tmp/ansible_log_demo.txt` exists
154+
- Tail the file manually to confirm updates:
155+
```bash
156+
tail -f /tmp/ansible_log_demo.txt
157+
```
158+
159+
- Adjust `Refresh_Interval` or restart Fluent Bit if needed

‎docs/integration/devops/github-actions.md

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
---
2+
3+
title: GitHub Actions Integration Guide
4+
description: Stream GitHub Actions workflow logs to OpenObserve using the `mdp/openobserve_github_action_logs` action.
5+
6+
---
7+
8+
# Integration with GitHub Actions
9+
10+
This guide explains how to stream **GitHub Actions workflow logs** to OpenObserve using a community-supported GitHub Action. Once integrated, logs from your CI/CD workflows are automatically sent to OpenObserve for monitoring and analysis.
11+
12+
## Overview
13+
14+
This integration captures metadata and logs from GitHub Actions jobs and pushes them directly to OpenObserve using the [mdp/openobserve_github_action_logs](https://github.com/mdp/openobserve_github_action_logs) Action. It requires minimal configuration and enables observability for CI pipelines.
15+
16+
## Steps to Integrate
17+
18+
??? "Prerequisites"
19+
20+
* OpenObserve account ([Cloud](https://cloud.openobserve.ai/web/) or [Self-Hosted](../../../quickstart/#self-hosted-installation))
21+
* GitHub repository
22+
23+
??? "Step 1: Get OpenObserve Credentials"
24+
25+
1. Log in to your OpenObserve instance
26+
2. Navigate to **Data Sources → custom → logs → curl**
27+
3. Copy the following values from the generated `curl` command:
28+
29+
From:
30+
31+
```bash
32+
curl -u [email protected]:18qlg4b673Rgdgd2 -k https://api.openobserve.ai/api/your_organization/<stream_name>/_json -d [JSON-DATA]
33+
```
34+
35+
Extract values:
36+
37+
```
38+
OPENOBSERVE_ENDPOINT : https://api.openobserve.ai/api/your_organization/<stream_name>/_json
39+
OPENOBSERVE_USERNAME : [email protected]
40+
OPENOBSERVE_KEY : 18qlg4b673Rgdgd2
41+
```
42+
43+
??? "Step 2: Add Secrets to GitHub Environment"
44+
45+
1. In your GitHub repository, go to **Settings → Environments**
46+
2. Click **New environment** and name it `dev`
47+
48+
![Create GitHub Environment](../images/devops/github-actions/dev-env.png){: style="height:500px"}
49+
3. Under the `dev` environment, click **Environment secrets → Add environment secret** and add the following: `OPENOBSERVE_USERNAME`, `OPENOBSERVE_KEY`, and `OPENOBSERVE_ENDPOINT` using the values from Step 1.
50+
51+
![Add Secrets to GitHub Environment](../images/devops/github-actions/add-secret.png){: style="height:300px"}
52+
53+
??? "Step 3: Add GitHub Action to Your Workflow"
54+
55+
Add the following to your GitHub Actions YAML file:
56+
57+
```yaml
58+
jobs:
59+
build-and-test:
60+
runs-on: ubuntu-latest
61+
steps:
62+
- uses: actions/checkout@v3
63+
- run: |
64+
npm ci
65+
- run: |
66+
npm run all
67+
- run: npm test
68+
69+
send_logs_to_openobserve:
70+
runs-on: ubuntu-latest
71+
needs: build-and-test # ensures logs are sent after main job
72+
environment: dev
73+
steps:
74+
- uses: actions/checkout@v3
75+
- uses: mdp/openobserve_github_action_logs@main
76+
with:
77+
openobserve_endpoint: ${{ secrets.OPENOBSERVE_ENDPOINT }}
78+
openobserve_username: ${{ secrets.OPENOBSERVE_USERNAME }}
79+
openobserve_key: ${{ secrets.OPENOBSERVE_KEY }}
80+
github_token: ${{ secrets.GITHUB_TOKEN }}
81+
```
82+
83+
Once this is in place, all logs from your GitHub Actions jobs will be automatically piped into OpenObserve.
84+
85+
> Note: The example uses Node.js-specific commands (npm ci, npm run all).
86+
If your project is not Node.js based, replace these steps with your own project's build/test commands (e.g., make, pytest, go test, etc.).
87+
88+
??? "Step 4: Verify Logs in OpenObserve"
89+
90+
1. In your Openobserve instance, Go to **Logs** → select your log stream → Set time range → Click **Run Query**
91+
92+
![Verify Logs](../images/devops/github-actions/verify-logs.png)
93+
94+
## Troubleshooting
95+
96+
??? "No logs appearing?"
97+
98+
* Double-check that the GitHub secrets were added under the correct **environment (`dev`)**
99+
* Ensure the `openobserve_endpoint` is correct and matches the `curl` example
100+
* Confirm the `send_logs_to_openobserve` job ran successfully in GitHub Actions
101+

0 commit comments

Comments
 (0)