Skip to content

Commit 94124d5

Browse files
gabriel-farachegabriel-farache
committed
Update docs
Signed-off-by: gabriel-farache <gfarache@redhat.com>
1 parent ebea34d commit 94124d5

File tree

10 files changed

+383
-6072
lines changed

10 files changed

+383
-6072
lines changed

‎content/main/docs/serverless-workflows/configuration/log-aggregation.md‎

Lines changed: 327 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,118 @@ export QUARKUS_EXTENSIONS="${QUARKUS_EXTENSIONS},io.quarkus:quarkus-logging-json
7070

7171
**Note**: This extension is **required** for JSON log formatting. Without it, logs will remain in plain text format.
7272

73+
## File-Based JSON Logging
74+
75+
In some deployment scenarios, you may need to output logs to a file instead of (or in addition to) the console. This is useful when:
76+
77+
- Using sidecar containers that read log files (e.g., Fluent Bit file input)
78+
- Integrating with legacy log collection systems that expect file-based logs
79+
- Debugging locally with persistent log files
80+
- Collecting logs from environments where stdout/stderr collection is limited
81+
82+
### Basic File Logging Configuration
83+
84+
Add the following properties to enable JSON logging to a file:
85+
86+
```properties
87+
# Enable file logging
88+
quarkus.log.file.enable=true
89+
quarkus.log.file.path=/var/log/sonataflow/workflow.log
90+
91+
# Enable JSON format for file output
92+
quarkus.log.file.json=true
93+
quarkus.log.file.json.pretty-print=false
94+
95+
# Include MDC context fields (processInstanceId, traceId, spanId)
96+
quarkus.log.file.json.print-details=true
97+
98+
# Set log level for file output
99+
quarkus.log.file.level=INFO
100+
```
101+
102+
### File Rotation Configuration
103+
104+
For production environments, configure log rotation to prevent disk space issues:
105+
106+
```properties
107+
# Enable file logging with rotation
108+
quarkus.log.file.enable=true
109+
quarkus.log.file.path=/var/log/sonataflow/workflow.log
110+
111+
# JSON format
112+
quarkus.log.file.json=true
113+
quarkus.log.file.json.pretty-print=false
114+
quarkus.log.file.json.print-details=true
115+
116+
# Rotation settings
117+
quarkus.log.file.rotation.max-file-size=10M
118+
quarkus.log.file.rotation.max-backup-index=5
119+
quarkus.log.file.rotation.file-suffix=.yyyy-MM-dd
120+
quarkus.log.file.rotation.rotate-on-boot=true
121+
```
122+
123+
This configuration:
124+
- Rotates logs when they reach 10MB
125+
- Keeps up to 5 backup files
126+
- Adds date suffix to rotated files
127+
- Rotates on application startup
128+
129+
### Combined Console and File Logging
130+
131+
You can enable both console and file JSON logging simultaneously:
132+
133+
```properties
134+
# Console JSON logging (for container log collectors)
135+
quarkus.log.console.json=true
136+
quarkus.log.console.json.pretty-print=false
137+
quarkus.log.console.json.print-details=true
138+
139+
# File JSON logging (for file-based collectors)
140+
quarkus.log.file.enable=true
141+
quarkus.log.file.path=/var/log/sonataflow/workflow.log
142+
quarkus.log.file.json=true
143+
quarkus.log.file.json.pretty-print=false
144+
quarkus.log.file.json.print-details=true
145+
quarkus.log.file.rotation.max-file-size=10M
146+
quarkus.log.file.rotation.max-backup-index=5
147+
```
148+
149+
### Kubernetes Volume Configuration
150+
151+
When using file-based logging in Kubernetes, ensure the log directory is properly mounted:
152+
153+
```yaml
154+
apiVersion: sonataflow.org/v1alpha08
155+
kind: SonataFlow
156+
metadata:
157+
name: my-workflow
158+
spec:
159+
podTemplate:
160+
container:
161+
volumeMounts:
162+
- name: logs
163+
mountPath: /var/log/sonataflow
164+
volumes:
165+
- name: logs
166+
emptyDir: {}
167+
```
168+
169+
For persistent logs or sidecar collection, use a shared volume:
170+
171+
```yaml
172+
spec:
173+
podTemplate:
174+
container:
175+
volumeMounts:
176+
- name: shared-logs
177+
mountPath: /var/log/sonataflow
178+
initContainers: []
179+
volumes:
180+
- name: shared-logs
181+
emptyDir:
182+
sizeLimit: 500Mi
183+
```
184+
73185
## Validation
74186
75187
Before deploying log aggregation, verify that JSON logging is working correctly:
@@ -186,7 +298,11 @@ helm install loki-stack grafana/loki-stack \
186298
187299
### Promtail Configuration
188300

189-
Configure Promtail to discover and parse SonataFlow logs:
301+
Configure Promtail to discover and parse SonataFlow logs. You can choose between scraping container stdout (default) or custom JSON log files.
302+
303+
#### Option A: Scrape Container Stdout (Default)
304+
305+
This configuration uses Kubernetes service discovery to collect logs from container stdout:
190306

191307
```yaml
192308
apiVersion: v1
@@ -238,6 +354,105 @@ data:
238354
traceId:
239355
```
240356
357+
#### Option B: Scrape JSON Log Files
358+
359+
When using [file-based JSON logging](#file-based-json-logging), configure Promtail as a sidecar to read from the shared log volume:
360+
361+
```yaml
362+
apiVersion: v1
363+
kind: ConfigMap
364+
metadata:
365+
name: promtail-sidecar-config
366+
namespace: sonataflow-infra
367+
data:
368+
config.yml: |
369+
server:
370+
http_listen_port: 3101
371+
372+
clients:
373+
- url: http://loki.sonataflow-observability.svc.cluster.local:3100/loki/api/v1/push
374+
375+
positions:
376+
filename: /var/log/positions.yaml
377+
378+
scrape_configs:
379+
- job_name: sonataflow-json-files
380+
static_configs:
381+
- targets:
382+
- localhost
383+
labels:
384+
job: sonataflow-workflows
385+
__path__: /var/log/sonataflow/*.log
386+
387+
pipeline_stages:
388+
- json:
389+
expressions:
390+
timestamp: timestamp
391+
level: level
392+
logger: loggerName
393+
message: message
394+
processInstanceId: mdc.processInstanceId
395+
traceId: mdc.traceId
396+
spanId: mdc.spanId
397+
398+
- labels:
399+
level:
400+
logger:
401+
processInstanceId:
402+
traceId:
403+
404+
- timestamp:
405+
source: timestamp
406+
format: RFC3339Nano
407+
```
408+
409+
**Promtail Sidecar Deployment:**
410+
411+
Add Promtail as a sidecar container in your SonataFlow CR:
412+
413+
```yaml
414+
apiVersion: sonataflow.org/v1alpha08
415+
kind: SonataFlow
416+
metadata:
417+
name: my-workflow
418+
namespace: sonataflow-infra
419+
spec:
420+
podTemplate:
421+
container:
422+
volumeMounts:
423+
- name: shared-logs
424+
mountPath: /var/log/sonataflow
425+
containers:
426+
- name: promtail-sidecar
427+
image: grafana/promtail:2.9.0
428+
args:
429+
- -config.file=/etc/promtail/config.yml
430+
volumeMounts:
431+
- name: shared-logs
432+
mountPath: /var/log/sonataflow
433+
readOnly: true
434+
- name: promtail-config
435+
mountPath: /etc/promtail
436+
- name: positions
437+
mountPath: /var/log
438+
resources:
439+
requests:
440+
cpu: 50m
441+
memory: 64Mi
442+
limits:
443+
cpu: 100m
444+
memory: 128Mi
445+
volumes:
446+
- name: shared-logs
447+
emptyDir:
448+
sizeLimit: 500Mi
449+
- name: promtail-config
450+
configMap:
451+
name: promtail-sidecar-config
452+
- name: positions
453+
emptyDir: {}
454+
```
455+
241456
### Query Examples
242457
243458
**Filter logs by process instance:**
@@ -287,6 +502,12 @@ helm install fluent-bit fluent/fluent-bit \
287502

288503
### Fluent Bit Configuration
289504

505+
You can configure Fluent Bit to collect logs from container stdout or from custom JSON log files.
506+
507+
#### Option A: Scrape Container Logs (Default)
508+
509+
This configuration collects logs from the standard Kubernetes container log location:
510+
290511
```yaml
291512
apiVersion: v1
292513
kind: ConfigMap
@@ -328,6 +549,111 @@ data:
328549
Logstash_Format On
329550
```
330551
552+
#### Option B: Scrape JSON Log Files (Sidecar)
553+
554+
When using [file-based JSON logging](#file-based-json-logging), deploy Fluent Bit as a sidecar to read from the shared log volume:
555+
556+
```yaml
557+
apiVersion: v1
558+
kind: ConfigMap
559+
metadata:
560+
name: fluent-bit-sidecar-config
561+
namespace: sonataflow-infra
562+
data:
563+
custom_parsers.conf: |
564+
[PARSER]
565+
Name sonataflow_json
566+
Format json
567+
Time_Key timestamp
568+
Time_Format %Y-%m-%dT%H:%M:%S.%L%z
569+
570+
fluent-bit.conf: |
571+
[SERVICE]
572+
Flush 1
573+
Log_Level info
574+
Parsers_File /fluent-bit/etc/custom_parsers.conf
575+
576+
[INPUT]
577+
Name tail
578+
Path /var/log/sonataflow/*.log
579+
Parser sonataflow_json
580+
Tag sonataflow.*
581+
Refresh_Interval 5
582+
Mem_Buf_Limit 50MB
583+
Read_from_Head True
584+
DB /var/log/flb_sonataflow.db
585+
586+
[FILTER]
587+
Name modify
588+
Match sonataflow.*
589+
Add kubernetes.namespace_name ${NAMESPACE}
590+
Add kubernetes.pod_name ${POD_NAME}
591+
592+
[OUTPUT]
593+
Name es
594+
Match sonataflow.*
595+
Host opensearch.logging.svc.cluster.local
596+
Port 9200
597+
Index sonataflow-logs-%Y.%m.%d
598+
Type _doc
599+
Logstash_Format On
600+
Retry_Limit 5
601+
```
602+
603+
**Fluent Bit Sidecar Deployment:**
604+
605+
Add Fluent Bit as a sidecar container in your SonataFlow CR:
606+
607+
```yaml
608+
apiVersion: sonataflow.org/v1alpha08
609+
kind: SonataFlow
610+
metadata:
611+
name: my-workflow
612+
namespace: sonataflow-infra
613+
spec:
614+
podTemplate:
615+
container:
616+
volumeMounts:
617+
- name: shared-logs
618+
mountPath: /var/log/sonataflow
619+
containers:
620+
- name: fluent-bit-sidecar
621+
image: fluent/fluent-bit:2.2
622+
env:
623+
- name: NAMESPACE
624+
valueFrom:
625+
fieldRef:
626+
fieldPath: metadata.namespace
627+
- name: POD_NAME
628+
valueFrom:
629+
fieldRef:
630+
fieldPath: metadata.name
631+
volumeMounts:
632+
- name: shared-logs
633+
mountPath: /var/log/sonataflow
634+
readOnly: true
635+
- name: fluent-bit-config
636+
mountPath: /fluent-bit/etc
637+
- name: fluent-bit-db
638+
mountPath: /var/log
639+
resources:
640+
requests:
641+
cpu: 50m
642+
memory: 64Mi
643+
limits:
644+
cpu: 100m
645+
memory: 128Mi
646+
volumes:
647+
- name: shared-logs
648+
emptyDir:
649+
sizeLimit: 500Mi
650+
- name: fluent-bit-config
651+
configMap:
652+
name: fluent-bit-sidecar-config
653+
- name: fluent-bit-db
654+
emptyDir: {}
655+
```
656+
331657
### OpenSearch Query Examples
332658
333659
**Process instance logs:**

0 commit comments

Comments
 (0)