You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/sources/operations/troubleshooting/troubleshoot-operations.md
+188-4Lines changed: 188 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1250,9 +1250,9 @@ The ring contains too many unhealthy instances to satisfy the replication factor
1250
1250
1251
1251
**Resolution:**
1252
1252
1253
-
1. **Check the health of ring members**:
1253
+
1. **Check the health of ring members**:
1254
1254
Open a browser and navigate to http://localhost:3100/ring. You should see the Loki ring page.
1255
-
1255
+
1256
1256
OR
1257
1257
1258
1258
```bash
@@ -1524,11 +1524,195 @@ After being disconnected from the memberlist cluster, the instance failed to rej
1524
1524
1525
1525
## Component readiness errors
1526
1526
1527
-
<!-- Additional content in next PRs. Just leaving the headings here for context and so that I can keep things in order if PRs merge out of sequence. -->
1527
+
Readiness errors occur when Loki components are not ready to serve requests. These errors are returned by the [`/ready` health check endpoint](http://localhost:3100/ready) and prevent load balancers from routing traffic to unready instances.
1528
1528
1529
-
## gRPC and message size errors
1529
+
### Error: Application is stopping
1530
+
1531
+
**Error message:**
1532
+
1533
+
```text
1534
+
Application is stopping
1535
+
```
1536
+
1537
+
**Cause:**
1538
+
1539
+
Loki is shutting down and no longer accepting new requests. This is normal during graceful shutdown.
1540
+
1541
+
**Resolution:**
1542
+
1543
+
1. **Wait for the instance to restart** if this is a rolling update.
1544
+
1. **Check if the shutdown is expected** (maintenance, scaling down).
1545
+
1. **Review orchestrator logs** (Kubernetes, systemd) if the shutdown is unexpected.
1546
+
1547
+
**Properties:**
1548
+
1549
+
- Enforced by: Loki readiness handler
1550
+
- Retryable: Yes (after restart)
1551
+
- HTTP status: 503 Service Unavailable
1552
+
- Configurable per tenant: No
1553
+
1554
+
### Error: Some services are not running
1555
+
1556
+
**Error message:**
1557
+
1558
+
```text
1559
+
Some services are not Running:
1560
+
<state>: <count>
1561
+
<state>: <count>
1562
+
```
1563
+
1564
+
For example:
1565
+
1566
+
```text
1567
+
Some services are not Running:
1568
+
Starting: 1
1569
+
Failed: 2
1570
+
```
1571
+
1572
+
**Cause:**
1573
+
1574
+
One or more internal Loki services have failed to start or have stopped unexpectedly. The error message lists each service state with a count of services in that state.
1575
+
1576
+
**Resolution:**
1577
+
1578
+
1. **Check Loki logs** for errors from the listed services.
1579
+
1. **Verify configuration** for the affected services.
<!-- Additional content in next PRs. Just leaving the headings here for context and so that I can keep things in order if PRs merge out of sequence. -->
0 commit comments