Skip to content

Commit 32597f2

Browse files
stevsmitstevsmit
authored
Merge pull request #39936 from aireilly/BZ2016216-PTP_scheduling
BZ2033449: Configuring FIFO priority scheduling for PTP hardware
2 parents 0dabc8b + d22e7b0 commit 32597f2

15 files changed

+128
-70
lines changed

‎modules/cnf-about-ptp-and-clock-synchronization.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
// Module included in the following assemblies:
22
//
3-
// * networking/configuring-ptp.adoc
3+
// * networking/using-ptp.adoc
44

55
[id="cnf-about-ptp-and-clock-synchronization_{context}"]
66
= About PTP and clock synchronization error events

‎modules/cnf-about-ptp-fast-event-notifications-framework.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
// Module included in the following assemblies:
22
//
3-
// * networking/configuring-ptp.adoc
3+
// * networking/using-ptp.adoc
44

55
[id="cnf-about-ptp-fast-event-notifications-framework_{context}"]
66
= About the PTP fast event notifications framework

‎modules/cnf-configuring-fifo-priority-scheduling-for-ptp.adoc

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * networking/using-ptp.adoc
4+
5+
[id="cnf-configuring-fifo-priority-scheduling-for-ptp_{context}"]
6+
= Configuring FIFO priority scheduling for PTP hardware
7+
8+
In telco or other deployment configurations that require low latency performance, PTP daemon threads run in a constrained CPU footprint alongside the rest of the infrastructure components. By default, PTP threads run with the `SCHED_OTHER` policy. Under high load, these threads might not get the scheduling latency they require for error-free operation.
9+
10+
To mitigate against potential scheduling latency errors, you can configure the PTP Operator `linuxptp` services to allow threads to run with a `SCHED_FIFO` policy. If `SCHED_FIFO` is set for a `PtpConfig` CR, then `ptp4l` and `phc2sys` will run in the parent container under `chrt` with a priority set by the `ptpSchedulingPriority` field of the `PtpConfig` CR.
11+
12+
[NOTE]
13+
====
14+
Setting `ptpSchedulingPolicy` is optional, and is only required if you are experiencing latency errors.
15+
====
16+
17+
.Procedure
18+
19+
. Edit the `PtpConfig` CR profile:
20+
+
21+
[source,terminal]
22+
----
23+
$ oc edit PtpConfig -n openshift-ptp
24+
----
25+
26+
. Change the `ptpSchedulingPolicy` and `ptpSchedulingPriority` fields:
27+
+
28+
[source,yaml]
29+
----
30+
apiVersion: ptp.openshift.io/v1
31+
kind: PtpConfig
32+
metadata:
33+
name: <ptp_config_name>
34+
namespace: openshift-ptp
35+
...
36+
spec:
37+
profile:
38+
- name: "profile1"
39+
...
40+
ptpSchedulingPolicy: SCHED_FIFO <1>
41+
ptpSchedulingPriority: 65 <2>
42+
----
43+
<1> Scheduling policy for `ptp4l` and `phc2sys` processes. Use `SCHED_FIFO` on systems that support FIFO scheduling.
44+
<2> Required. Sets the integer value 1-65 used to configure FIFO priority for `ptp4l` and `phc2sys` processes.
45+
46+
. Save and exit to apply the changes to the `PtpConfig` CR.
47+
48+
.Verification
49+
50+
. Get the name of the `linuxptp-daemon` pod and corresponding node where the `PtpConfig` CR has been applied:
51+
+
52+
[source,terminal]
53+
----
54+
$ oc get pods -n openshift-ptp -o wide
55+
----
56+
+
57+
.Example output
58+
[source,terminal]
59+
----
60+
NAME READY STATUS RESTARTS AGE IP NODE
61+
linuxptp-daemon-gmv2n 3/3 Running 0 1d17h 10.1.196.24 compute-0.example.com
62+
linuxptp-daemon-lgm55 3/3 Running 0 1d17h 10.1.196.25 compute-1.example.com
63+
ptp-operator-3r4dcvf7f4-zndk7 1/1 Running 0 1d7h 10.129.0.61 control-plane-1.example.com
64+
----
65+
66+
. Check that the `ptp4l` process is running with the updated `chrt` FIFO priority:
67+
+
68+
[source,terminal]
69+
----
70+
$ oc -n openshift-ptp logs linuxptp-daemon-lgm55 -c linuxptp-daemon-container|grep chrt
71+
----
72+
+
73+
.Example output
74+
[source,terminal]
75+
----
76+
I1216 19:24:57.091872 1600715 daemon.go:285] /bin/chrt -f 65 /usr/sbin/ptp4l -f /var/run/ptp4l.0.config -2 --summary_interval -4 -m
77+
----
78+
79+

‎modules/cnf-configuring-the-ptp-fast-event-publisher.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
// Module included in the following assemblies:
22
//
3-
// * networking/configuring-ptp.adoc
3+
// * networking/using-ptp.adoc
44

55
[id="cnf-configuring-the-ptp-fast-event-publisher_{context}"]
66
= Configuring the PTP fast event notifications publisher

‎modules/cnf-fast-event-notifications-api-refererence.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
// Module included in the following assemblies:
22
//
3-
// * networking/configuring-ptp.adoc
3+
// * networking/using-ptp.adoc
44

55
[id="cnf-fast-event-notifications-api-refererence_{context}"]
66
= PTP fast event notifications REST API reference

‎modules/cnf-installing-amq-interconnect-messaging-bus.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
// Module included in the following assemblies:
22
//
3-
// * networking/configuring-ptp.adoc
3+
// * networking/using-ptp.adoc
44

55
[id="cnf-installing-amq-interconnect-messaging-bus_{context}"]
66
= Installing the AMQ messaging bus

‎modules/cnf-monitoring-fast-events-metrics-using-cli.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
// Module included in the following assemblies:
22
//
3-
// * networking/configuring-ptp.adoc
3+
// * networking/using-ptp.adoc
44

55
[id="cnf-monitoring-fast-events-metrics-using-cli_{context}"]
66
= Monitoring PTP fast event metrics using the CLI

‎modules/cnf-ptp-fast-event-metrics-in-prometheus.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
// Module included in the following assemblies:
22
//
3-
// * networking/configuring-ptp.adoc
3+
// * networking/using-ptp.adoc
44

55
[id="cnf-fast-event-metrics-in-prometheus_{context}"]
66
= Monitoring PTP fast event metrics in the web console

‎modules/cnf-troubleshooting-common-ptp-operator-issues.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
// Module included in the following assemblies:
22
//
3-
// * networking/configuring-ptp.adoc
3+
// * networking/using-ptp.adoc
44

55
[id="cnf-troubleshooting-common-ptp-operator-issues_{context}"]
66
= Troubleshooting common PTP Operator issues

‎modules/nw-ptp-configuring-linuxptp-services-as-boundary-clock.adoc

Lines changed: 18 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
// Module included in the following assemblies:
22
//
3-
// * networking/multiple_networks/configuring-ptp.adoc
3+
// * networking/using-ptp.adoc
44

55
[id="configuring-linuxptp-services-as-boundary-clock_{context}"]
66
= Configuring linuxptp services as boundary clock
@@ -137,12 +137,14 @@ spec:
137137
userDescription ;
138138
timeSource 0xA0
139139
phc2sysOpts: "-a -r" <9>
140-
recommend: <10>
141-
- profile: "profile1" <11>
142-
priority: 10 <12>
143-
match: <13>
144-
- nodeLabel: "node-role.kubernetes.io/worker" <14>
145-
nodeName: "compute-0.example.com" <15>
140+
ptpSchedulingPolicy: SCHED_OTHER <10>
141+
ptpSchedulingPriority: 65 <11>
142+
recommend: <12>
143+
- profile: "profile1" <13>
144+
priority: 10 <14>
145+
match: <15>
146+
- nodeLabel: "node-role.kubernetes.io/worker" <16>
147+
nodeName: "compute-0.example.com" <17>
146148
----
147149
<1> The name of the `PtpConfig` CR.
148150
<2> Specify an array of one or more `profile` objects.
@@ -153,12 +155,14 @@ spec:
153155
<7> The interface name to synchronize from.
154156
<8> The interface to synchronize devices connected to the interface.
155157
<9> Specify system config options for the `phc2sys` service, for example `-a -r`. If this field is empty the PTP Operator does not start the `phc2sys` service.
156-
<10> Specify an array of one or more `recommend` objects which define rules on how the `profile` should be applied to nodes.
157-
<11> Specify the `profile` object name defined in the `profile` section.
158-
<12> Specify the `priority` with an integer value between `0` and `99`. A larger number gets lower priority, so a priority of `99` is lower than a priority of `10`. If a node can be matched with multiple profiles according to rules defined in the `match` field, the profile with the higher priority is applied to that node.
159-
<13> Specify `match` rules with `nodeLabel` or `nodeName`.
160-
<14> Specify `nodeLabel` with the `key` of `node.Labels` from the node object.
161-
<15> Specify `nodeName` with `node.Name` from the node object.
158+
<10> Scheduling policy for ptp4l and phc2sys processes. Default value is `SCHED_OTHER`. Use `SCHED_FIFO` on systems that support FIFO scheduling.
159+
<11> Integer value from 1-65 used to set FIFO priority for `ptp4l` and `phc2sys` processes. Required if `SCHED_FIFO` is set for `ptpSchedulingPolicy`.
160+
<12> Specify an array of one or more `recommend` objects that define rules on how the `profile` should be applied to nodes.
161+
<13> Specify the `profile` object name defined in the `profile` section.
162+
<14> Specify the `priority` with an integer value between `0` and `99`. A larger number gets lower priority, so a priority of `99` is lower than a priority of `10`. If a node can be matched with multiple profiles according to rules defined in the `match` field, the profile with the higher priority is applied to that node.
163+
<15> Specify `match` rules with `nodeLabel` or `nodeName`.
164+
<16> Specify `nodeLabel` with the `key` of `node.Labels` from the node object.
165+
<17> Specify `nodeName` with `node.Name` from the node object.
162166

163167
. Create the CR by running the following command:
164168
+
@@ -169,7 +173,7 @@ $ oc create -f boundary-clock-ptp-config.yaml
169173

170174
.Verification steps
171175

172-
. Check that the `PtpConfig` profile is applied to node.
176+
. Check that the `PtpConfig` profile is applied to the node.
173177

174178
.. Get the list of pods in the `openshift-ptp` namespace by running the following command:
175179
+

0 commit comments

Comments
 (0)