-
Notifications
You must be signed in to change notification settings - Fork 67
Description
What steps will reproduce the bug?
I am deploying a NiFi cluster using the NiFiKop operator on [AWS/Azure/Local] Kubernetes. The NiFi pods are entering a continuous restart loop. Based on the logs, NiFi is still in the process of unpacking NAR files and initializing the Jetty server when Kubernetes/NiFiKop triggers a shutdown because the health checks (LivenessProbes) fail.
Environment:
NiFi Version: 1.23.2
Storage Type: [e.g., EBS gp3]
The Problem: NiFi 1.23.2 takes approximately 5-8 minutes to fully start in my environment. The default livenessProbe settings in the operator (or the inability to override them in my CRD version) cause Kubernetes to kill the pod before it reaches a "Ready" state.
Logs Observed:
[Standard NiFi logs showing NAR extraction...]
INFO [main] o.a.n.controller.StandardFlowService...
INFO [Thread-1] org.apache.nifi.NiFi NiFi PID [1] shutdown started...
Received SIGTERM, shutting down gracefullly.
What I have tried:
Adding livenessProbe overrides in nodeConfigGroups (Rejected by CRD or ignored).
Manually patching the pod (Operator reverts the change).
Increasing resources (CPU/Memory), but the unpacking time still exceeds the probe timeout.
Question: How can I correctly override the initialDelaySeconds for liveness and readiness probes in this version of the operator to allow for a 600-900 second startup time? Is there a specific CRD path I should be using for NiFi 1.2x?
I am stuck here , Please help me out of this cycle
What is the expected behavior?
Pods should be up and running and I should able to open UI
What do you see instead?
Pods are restarting for every 10 - 15 seconds and its noy memory issue I am using t3.Xlarge aws instance
Possible solution
No response
NiFiKop version
v1.16.0
Golang version
1.19
Kubernetes version
Client Version: v1.35.0
Kustomize Version: v5.7.1
Server Version: v1.34.3+k3s1
NiFi version
1.23
Additional context
No response