Skip to content

Commit 0bf9ed5

Browse files
committed
Telcodocs-1038 updating IRQ lb guidance
1 parent 40ba61b commit 0bf9ed5

File tree

3 files changed

+68
-16
lines changed

3 files changed

+68
-16
lines changed
Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
// Module included in the following assemblies:
2+
//
3+
// scalability_and_performance/cnf-low-latency-tuning.adoc
4+
5+
:_content-type: CONCEPT
6+
[id="about_irq_affinity_setting_{context}"]
7+
= About support of IRQ affinity setting
8+
9+
Some IRQ controllers lack support for IRQ affinity setting and will always expose all online CPUs as the IRQ mask. These IRQ controllers effectively run on CPU 0.
10+
11+
The following are examples of drivers and hardware that Red Hat are aware lack support for IRQ affinity setting. The list is, by no means, exhaustive:
12+
13+
* Some RAID controller drivers, such as `megaraid_sas`
14+
* Many non-volatile memory express (NVMe) drivers
15+
* Some LAN on motherboard (LOM) network controllers
16+
* The driver uses `managed_irqs`
17+
18+
[NOTE]
19+
====
20+
The reason they do not support IRQ affinity setting might be associated with factors such as the type of processor, the IRQ controller, or the circuitry connections in the motherboard.
21+
====
22+
23+
If the effective affinity of any IRQ is set to an isolated CPU, it might be a sign of some hardware or driver not supporting IRQ affinity setting. To find the effective affinity, log in to the host and run the following command:
24+
25+
[source,terminal]
26+
----
27+
$ find /proc/irq/ -name effective_affinity -exec sh -c 'i="$1"; mask=$(cat $i); file=$(echo $i); echo $file: $mask' _ {} \;
28+
----
29+
30+
.Example output
31+
32+
[source,terminal]
33+
----
34+
/proc/irq/0/effective_affinity: 1
35+
/proc/irq/1/effective_affinity: 8
36+
/proc/irq/2/effective_affinity: 0
37+
/proc/irq/3/effective_affinity: 1
38+
/proc/irq/4/effective_affinity: 2
39+
/proc/irq/5/effective_affinity: 1
40+
/proc/irq/6/effective_affinity: 1
41+
/proc/irq/7/effective_affinity: 1
42+
/proc/irq/8/effective_affinity: 1
43+
/proc/irq/9/effective_affinity: 2
44+
/proc/irq/10/effective_affinity: 1
45+
/proc/irq/11/effective_affinity: 1
46+
/proc/irq/12/effective_affinity: 4
47+
/proc/irq/13/effective_affinity: 1
48+
/proc/irq/14/effective_affinity: 1
49+
/proc/irq/15/effective_affinity: 1
50+
/proc/irq/24/effective_affinity: 2
51+
/proc/irq/25/effective_affinity: 4
52+
/proc/irq/26/effective_affinity: 2
53+
/proc/irq/27/effective_affinity: 1
54+
/proc/irq/28/effective_affinity: 8
55+
/proc/irq/29/effective_affinity: 4
56+
/proc/irq/30/effective_affinity: 4
57+
/proc/irq/31/effective_affinity: 8
58+
/proc/irq/32/effective_affinity: 8
59+
/proc/irq/33/effective_affinity: 1
60+
/proc/irq/34/effective_affinity: 2
61+
----
62+
63+
Some drivers use `managed_irqs`, whose affinity is managed internally by the kernel and userspace cannot change the affinity. In some cases, these IRQs might be assigned to isolated CPUs. For more information about `managed_irqs`, see link:https://access.redhat.com/solutions/4819541[Affinity of managed interrupts cannot be changed even if they target isolated CPU].

modules/cnf-configure_for_irq_dynamic_load_balancing.adoc

Lines changed: 4 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -2,14 +2,15 @@
22
//
33
// scalability_and_performance/cnf-low-latency-tuning.adoc
44

5+
:_content-type: PROCEDURE
56
[id="configuring_for_irq_dynamic_load_balancing_{context}"]
67
= Configuring a node for IRQ dynamic load balancing
78

89
Configure a cluster node for IRQ dynamic load balancing to control which cores can receive device interrupt requests (IRQ).
910

1011
.Prerequisites
1112

12-
* For core isolation, all server hardware components must support IRQ affinity. For more information, see the _Additional resources_ section.
13+
* For core isolation, all server hardware components must support IRQ affinity. To check if the hardware components of your server support IRQ affinity, view the server's hardware specifications or contact your hardware provider.
1314
1415
.Procedure
1516

@@ -94,7 +95,7 @@ $ oc exec -it dynamic-irq-pod -- /bin/bash -c "grep Cpus_allowed_list /proc/self
9495
----
9596
Cpus_allowed_list: 2-3
9697
----
97-
. Ensure the node configuration is applied correctly. SSH into the node to verify the configuration.
98+
. Ensure the node configuration is applied correctly. Log in to the node to verify the configuration.
9899
+
99100
[source,terminal]
100101
----
@@ -175,16 +176,4 @@ find /proc/irq/ -name smp_affinity_list -exec sh -c 'i="$1"; mask=$(cat $i); fil
175176
/proc/irq/28/smp_affinity_list: 1
176177
/proc/irq/29/smp_affinity_list: 0
177178
/proc/irq/30/smp_affinity_list: 0-5
178-
----
179-
180-
Some IRQ controllers do not support IRQ re-balancing and will always expose all online CPUs as the IRQ mask. These IRQ controllers effectively run on CPU 0. For more information on the host configuration, SSH into the host and run the following, replacing `<irq-num>` with the CPU number that you want to query:
181-
182-
[source,terminal]
183-
----
184-
$ cat /proc/irq/<irq-num>/effective_affinity
185-
----
186-
187-
[role="_additional-resources"]
188-
.Additional resources
189-
190-
* xref:../scalability_and_performance/cnf-low-latency-tuning.adoc#ref_hardware-compatibility-with-irq-affinity_cnf-master[Hardware compatibility with IRQ affinity]
179+
----

scalability_and_performance/cnf-low-latency-tuning.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ include::modules/cnf-allocating-multiple-huge-page-sizes.adoc[leveloffset=+2]
3939

4040
include::modules/cnf-configure_for_irq_dynamic_load_balancing.adoc[leveloffset=+2]
4141

42-
include::modules/ref_hardware-compatibility-with-irq-affinity.adoc[leveloffset=+3]
42+
include::modules/cnf-about-irq-affinity-setting.adoc[leveloffset=+2]
4343

4444
include::modules/configuring_hyperthreading_for_a_cluster.adoc[leveloffset=+2]
4545

0 commit comments

Comments
 (0)