Skip to content

Commit 9117b88

Browse files
rborbo
committed
Added domain.xml adjustment example via sidecar hook
Fix #60
1 parent 3dee0ce commit 9117b88

File tree

2 files changed

+278
-0
lines changed

2 files changed

+278
-0
lines changed

‎content/kubevirt/adjust-domain-xml.md‎

Lines changed: 277 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,277 @@
1+
---
2+
title: Adjust domain.xml
3+
linktitle: Adjust domain.xml
4+
description: Adjust domain.xml via sidecar hook to add a WWN to a disk
5+
tags: ['v4.20','sidecarhook','kubevirt', 'libvirt']
6+
---
7+
# Adjust domain.xml to add a WWN to a disk
8+
9+
This guide demonstrates how to use KubeVirt sidecar hooks to modify the libvirt domain.xml and add a World Wide Name (WWN) to a disk. This is particularly useful when testing IBM Fusion Access for SAN, which requires disks to have a specific WWN.
10+
11+
## Prerequisites
12+
13+
Documentation:
14+
15+
- [Kubevirt Feature Gates](https://github.com/kubevirt/hyperconverged-cluster-operator/blob/main/docs/cluster-configuration.md#kubevirt-feature-gates)
16+
- [Hook Sidecar Container](https://kubevirt.io/user-guide/user_workloads/hook-sidecar/#configmap-with-python-script)
17+
18+
Tested with:
19+
20+
|Component|Version|
21+
|---|---|
22+
|OpenShift|v4.20.12|
23+
|OpenShift Virt|v4.20.3|
24+
25+
## Problem Statement
26+
27+
To test IBM Fusion Access for SAN, the attached disk needs a WWN (World Wide Name). The libvirt domain.xml must be patched to include this WWN in the disk configuration.
28+
29+
## Current Disk Configuration
30+
31+
The current domain.xml contains a disk without a WWN:
32+
33+
```xml
34+
<disk type='block' device='disk'>
35+
<driver name='qemu' type='raw' cache='none' error_policy='stop' io='native' discard='unmap'/>
36+
<source dev='/var/run/kubevirt/hotplug-disks/ibm-fusion-disk' index='1'/>
37+
<backingStore/>
38+
<target dev='sda' bus='scsi'/>
39+
<shareable/>
40+
<alias name='ua-ibm-fusion-disk'/>
41+
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
42+
</disk>
43+
```
44+
45+
## Desired Disk Configuration
46+
47+
We need to add `<wwn>5000c500155a3456</wwn>` to the disk configuration. The target configuration should look like this:
48+
49+
```xml
50+
<disk type='block' device='disk'>
51+
<driver name='qemu' type='raw' cache='none'/>
52+
<source dev='/dev/mapper/vg0-fusion' index='2'/>
53+
<backingStore/>
54+
<target dev='sdb' bus='scsi'/>
55+
<wwn>5000c500155a3456</wwn>
56+
<alias name='scsi0-0-0-1'/>
57+
<address type='drive' controller='0' bus='0' target='0' unit='1'/>
58+
</disk>
59+
```
60+
61+
## Step 1: Enable the Sidecar Feature Gate
62+
63+
The Sidecar feature gate must be enabled to use hook sidecar containers. For more information, see the [Kubevirt Feature Gates documentation](https://github.com/kubevirt/hyperconverged-cluster-operator/blob/main/docs/cluster-configuration.md#kubevirt-feature-gates).
64+
65+
Edit the HyperConverged operator annotation using `oc annotate`:
66+
67+
```shell
68+
oc annotate hco -n openshift-cnv kubevirt-hyperconverged \
69+
kubevirt.kubevirt.io/jsonpatch='[ {"op": "add", "path": "/spec/configuration/developerConfiguration/featureGates/-","value": "Sidecar"} ]'
70+
```
71+
72+
Alternatively, you can edit the HCO resource directly:
73+
74+
```shell
75+
oc edit hco -n openshift-cnv kubevirt-hyperconverged
76+
```
77+
78+
Add the following annotation:
79+
80+
```yaml
81+
kubevirt.kubevirt.io/jsonpatch: '[ {"op": "add", "path": "/spec/configuration/developerConfiguration/featureGates/-","value": "Sidecar"} ]'
82+
```
83+
84+
Verify that the feature gate is enabled:
85+
86+
```shell
87+
oc get kubevirt -n openshift-cnv kubevirt-kubevirt-hyperconverged \
88+
-o jsonpath="{.spec.configuration.developerConfiguration.featureGates}" | jq
89+
```
90+
91+
Expected output:
92+
93+
```json
94+
[
95+
...
96+
"Sidecar"
97+
]
98+
```
99+
100+
## Step 2: Create a Sidecar Hook to Inspect domain.xml
101+
102+
!!! tip
103+
104+
The domain.xml format differs from the running configuration. During hook execution, there are no newlines in the domain.xml, making it a single-line XML string. This is important to consider when developing your modification script.
105+
106+
### Create a ConfigMap for the Hook Script
107+
108+
Create a ConfigMap in the same namespace as your VirtualMachine:
109+
110+
```yaml
111+
apiVersion: v1
112+
kind: ConfigMap
113+
metadata:
114+
name: add-wwn
115+
data:
116+
add-wwn.sh: |
117+
#!/bin/sh
118+
tempFile=`mktemp --dry-run`
119+
echo $tempFile >> /tmp/onDefineDomain-tmpfile.log
120+
echo $4 > $tempFile
121+
cat $tempFile
122+
```
123+
124+
Save this as `add-wwn-configmap.yaml` and apply it:
125+
126+
```shell
127+
oc apply -f add-wwn-configmap.yaml
128+
```
129+
130+
### Add Hook Sidecar Annotation to Your VirtualMachine
131+
132+
Add the `hooks.kubevirt.io/hookSidecars` annotation to your VirtualMachine specification:
133+
134+
```yaml
135+
spec:
136+
template:
137+
metadata:
138+
annotations:
139+
hooks.kubevirt.io/hookSidecars: >
140+
[
141+
{
142+
"args": ["--version", "v1alpha2"],
143+
"configMap": {"name": "add-wwn", "key": "add-wwn.sh", "hookPath": "/usr/bin/onDefineDomain"}
144+
}
145+
]
146+
```
147+
148+
### Start the Virtual Machine and Inspect domain.xml
149+
150+
Start the virtual machine:
151+
152+
```shell
153+
oc apply -f your-vm.yaml
154+
```
155+
156+
Wait for the VM to start, then connect to the `hook-sidecar-0` container of the virt-launcher pod:
157+
158+
```shell
159+
oc rsh -c hook-sidecar-0 virt-launcher-<vm-name>-<pod-id>
160+
```
161+
162+
Inside the container, check the log file to see the temporary files created:
163+
164+
```shell
165+
cat /tmp/onDefineDomain-tmpfile.log
166+
```
167+
168+
Example output:
169+
170+
```
171+
/tmp/tmp.Q7YIeHv8cV
172+
/tmp/tmp.XvyR8QbHTi
173+
```
174+
175+
Review the XML content in one of these temporary files:
176+
177+
```shell
178+
cat /tmp/tmp.XvyR8QbHTi
179+
```
180+
181+
This will show you the actual format of the domain.xml that the hook receives. Use this to develop your `sed` command for modifying the XML.
182+
183+
## Step 3: Update the ConfigMap with the sed Command
184+
185+
Now that you've inspected the domain.xml format, update the ConfigMap with the `sed` command to add the WWN. The script reads the domain.xml from `$4`, modifies it, and outputs the modified XML.
186+
187+
```yaml
188+
apiVersion: v1
189+
kind: ConfigMap
190+
metadata:
191+
name: add-wwn
192+
data:
193+
add-wwn.sh: |
194+
#!/bin/sh
195+
tempFile=`mktemp --dry-run`
196+
echo $tempFile >> /tmp/onDefineDomain-tmpfile.log
197+
echo $4 > $tempFile
198+
sed -i "s|<shareable></shareable>|<shareable></shareable><wwn>5000c500155a3456</wwn>|" $tempFile
199+
cat $tempFile
200+
```
201+
202+
**Explanation of the sed command:**
203+
204+
- `sed -i` - Edit the file in place
205+
- `s|<shareable></shareable>|<shareable></shareable><wwn>5000c500155a3456</wwn>|` - Substitute the `<shareable></shareable>` tag with the same tag followed by the WWN element
206+
207+
Apply the updated ConfigMap:
208+
209+
```shell
210+
oc apply -f add-wwn-configmap.yaml
211+
```
212+
213+
Restart the VirtualMachine to apply the changes:
214+
215+
```shell
216+
oc apply -f your-vm.yaml
217+
virtctl restart <your-vm-name>
218+
```
219+
220+
## Step 4: Verify the WWN
221+
222+
Once the VM has restarted, connect to it and verify that the disk has the WWN:
223+
224+
```shell
225+
virtctl ssh <your-vm-name> or ssh ...
226+
```
227+
228+
Inside the VM, check the disk WWN:
229+
230+
```shell
231+
lsblk -o +WWN
232+
```
233+
234+
Expected output:
235+
236+
```
237+
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS WWN
238+
loop0 7:0 0 5.8M 1 loop
239+
sda 8:0 0 500G 0 disk 0x5000c500155a3456
240+
vda 252:0 0 120G 0 disk
241+
├─vda1 252:1 0 1M 0 part
242+
├─vda2 252:2 0 127M 0 part
243+
├─vda3 252:3 0 384M 0 part /boot
244+
└─vda4 252:4 0 119.5G 0 part /var
245+
/sysroot/ostree/deploy/rhcos/var
246+
/etc
247+
/sysroot
248+
```
249+
250+
The disk `sda` should now show the WWN `0x5000c500155a3456`.
251+
252+
## Troubleshooting
253+
254+
If the WWN is not appearing:
255+
256+
1. **Check the hook sidecar logs:**
257+
```shell
258+
oc logs -c hook-sidecar-0 virt-launcher-<vm-name>-<pod-id>
259+
```
260+
261+
2. **Verify the sed pattern matches your domain.xml:**
262+
- Connect to the hook-sidecar-0 container and check the actual XML format
263+
- Adjust the sed pattern if your XML structure differs
264+
265+
3. **Ensure the feature gate is enabled:**
266+
```shell
267+
oc get kubevirt -n openshift-cnv kubevirt-kubevirt-hyperconverged \
268+
-o jsonpath="{.spec.configuration.developerConfiguration.featureGates}"
269+
```
270+
271+
4. **Check that the ConfigMap is correctly referenced:**
272+
- Verify the ConfigMap name and namespace match your VM annotation
273+
- Ensure the key name matches (`add-wwn.sh`)
274+
275+
## Summary
276+
277+
You have successfully configured a KubeVirt sidecar hook to modify the libvirt domain.xml and add a WWN to a disk. This enables testing with IBM Fusion Access for SAN, which requires disks to have specific WWNs.

‎mkdocs.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -283,6 +283,7 @@ nav:
283283
- Networking: kubevirt/networking/index.md
284284
- Storage: kubevirt/storage.md
285285
- NFS CSI Driver: kubevirt/nfs-csi-driver.md
286+
- Adjust domain.xml: kubevirt/adjust-domain-xml.md
286287

287288
- Certificates: certificate.md
288289

0 commit comments

Comments
 (0)