Skip to content

Commit 3fab0ae

Browse files
committed
Add known issue for longhorn oversized volumes in v1.4.3
Harvester v1.4.3 updates Longhorn to v1.7.3, which has a problem with oversized volumes being marked "Not Ready" and unable to be deleted. I've added a note about this to the upgrade docs. This issue is also present in Harvester v1.5.0, and has already been documented, but that workaround refers to "/webhooks/0/rules/17", which doesn't work for LH v1.7.3 (it's rule 14 in that case), so I've updated the workaround to be a bit more generic and not depend on a specific rule index. Signed-off-by: Tim Serong <[email protected]>
1 parent d99e4d5 commit 3fab0ae

File tree

4 files changed

+79
-2
lines changed

4 files changed

+79
-2
lines changed

docs/upgrade/v1-4-1-to-v1-4-3.md

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -108,3 +108,38 @@ To fix the issue, perform any of the following actions:
108108

109109
Related issues:
110110
- [[BUG] AirGap Upgrades Seem Blocked with Fluentbit/FluentD](https://github.com/harvester/harvester/issues/7955)
111+
112+
### 2. Oversized Volumes
113+
114+
In Harvester v1.4.3, which uses Longhorn v1.7.3, oversized volumes (for example, 999999 Gi in size) are marked **Not Ready** and cannot be deleted.
115+
116+
To resolve this issue, perform the following steps:
117+
118+
1. Temporarily remove the PVC webhook rule.
119+
120+
```bash
121+
RULE_INDEX=$(kubectl get \
122+
validatingwebhookconfiguration longhorn-webhook-validator -o json \
123+
| jq '.webhooks[0].rules | map(.resources[0] == "persistentvolumeclaims") | index(true)')
124+
125+
kubectl patch validatingwebhookconfiguration longhorn-webhook-validator \
126+
--type='json' \
127+
-p="[{'op': 'remove', 'path': '/webhooks/0/rules/$RULE_INDEX'}]"
128+
```
129+
130+
1. Wait for the related PVC to be deleted.
131+
132+
1. Restore the PVC webhook rule to re-enable validation.
133+
134+
```bash
135+
kubectl patch validatingwebhookconfiguration longhorn-webhook-validator \
136+
--type='json' \
137+
-p='[{"op": "add", "path": "/webhooks/0/rules/-", "value": {"apiGroups":[""],"apiVersions":["v1"],"operations":["UPDATE"],"resources":["persistentvolumeclaims"],"scope":"Namespaced"}}]'
138+
```
139+
140+
The issue will be addressed in Longhorn v1.8.2, which will likely be included in Harvester v1.5.1.
141+
142+
Related issues:
143+
- Harvester: [Issue #8096](https://github.com/harvester/harvester/issues/8096)
144+
- Longhorn: [Issue #10741](https://github.com/longhorn/longhorn/issues/10741)
145+

docs/volume/create-volume.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,9 +38,13 @@ To resolve this issue, perform the following steps:
3838
1. Temporarily remove the PVC webhook rule.
3939

4040
```bash
41+
RULE_INDEX=$(kubectl get \
42+
validatingwebhookconfiguration longhorn-webhook-validator -o json \
43+
| jq '.webhooks[0].rules | map(.resources[0] == "persistentvolumeclaims") | index(true)')
44+
4145
kubectl patch validatingwebhookconfiguration longhorn-webhook-validator \
4246
--type='json' \
43-
-p='[{"op": "remove", "path": "/webhooks/0/rules/17"}]'
47+
-p="[{'op': 'remove', 'path': '/webhooks/0/rules/$RULE_INDEX'}]"
4448
```
4549

4650
1. Wait for the related PVC to be deleted.

versioned_docs/version-v1.4/volume/create-volume.md

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,40 @@ description: Create a volume from the Volume page.
2929

3030
![create-empty-volume](/img/v1.2/volume/create-empty-volume.png)
3131

32+
### Oversized Volumes
33+
34+
In Harvester v1.4.3, which uses Longhorn v1.7.3, oversized volumes (for example, 999999 Gi in size) are marked **Not Ready** and cannot be deleted.
35+
36+
To resolve this issue, perform the following steps:
37+
38+
1. Temporarily remove the PVC webhook rule.
39+
40+
```bash
41+
RULE_INDEX=$(kubectl get \
42+
validatingwebhookconfiguration longhorn-webhook-validator -o json \
43+
| jq '.webhooks[0].rules | map(.resources[0] == "persistentvolumeclaims") | index(true)')
44+
45+
kubectl patch validatingwebhookconfiguration longhorn-webhook-validator \
46+
--type='json' \
47+
-p="[{'op': 'remove', 'path': '/webhooks/0/rules/$RULE_INDEX'}]"
48+
```
49+
50+
1. Wait for the related PVC to be deleted.
51+
52+
1. Restore the PVC webhook rule to re-enable validation.
53+
54+
```bash
55+
kubectl patch validatingwebhookconfiguration longhorn-webhook-validator \
56+
--type='json' \
57+
-p='[{"op": "add", "path": "/webhooks/0/rules/-", "value": {"apiGroups":[""],"apiVersions":["v1"],"operations":["UPDATE"],"resources":["persistentvolumeclaims"],"scope":"Namespaced"}}]'
58+
```
59+
60+
The issue will be addressed in Longhorn v1.8.2, which will likely be included in Harvester v1.5.1.
61+
62+
Related issues:
63+
- Harvester: [Issue #8096](https://github.com/harvester/harvester/issues/8096)
64+
- Longhorn: [Issue #10741](https://github.com/longhorn/longhorn/issues/10741)
65+
3266
</TabItem>
3367
<TabItem value="api" label="API">
3468

versioned_docs/version-v1.5/volume/create-volume.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,9 +38,13 @@ To resolve this issue, perform the following steps:
3838
1. Temporarily remove the PVC webhook rule.
3939

4040
```bash
41+
RULE_INDEX=$(kubectl get \
42+
validatingwebhookconfiguration longhorn-webhook-validator -o json \
43+
| jq '.webhooks[0].rules | map(.resources[0] == "persistentvolumeclaims") | index(true)')
44+
4145
kubectl patch validatingwebhookconfiguration longhorn-webhook-validator \
4246
--type='json' \
43-
-p='[{"op": "remove", "path": "/webhooks/0/rules/17"}]'
47+
-p="[{'op': 'remove', 'path': '/webhooks/0/rules/$RULE_INDEX'}]"
4448
```
4549

4650
1. Wait for the related PVC to be deleted.

0 commit comments

Comments
 (0)