Two-step Mode Configuration

[freeman94]

This may be more of a question rather than an issue per se, but I am wondering if there is a way to configure sfptpd in master mode to only use two-step synchronization. I've looked through the docs and done a cursory search through the codebase, but it doesn't seem like there is.

For context, I'm trying to serve PTP to a networking switch which I have configured as a boundary clock. The host that is running sfptpd is configured with a LACP bond to the switch. Unfortunately, it seems like my switch only supports two-step mode. By default, it seems like sfptpd will start in one-step mode, however, I've noticed that on certain occasions sfptpd will operate in two-step mode. Perhaps this is under the condition that the switch is sending Announce packets which indicate it is using two-step mode? (a condition that would occur if sfptpd starts up after the switch, because the switch initially assumes the master role). Otherwise, I'm not sure what's going on so any clarification would be appreciated.

Either way, if I could configure sfptpd to always use two-step, I think I could resolve my issue.

[freeman94]

Here's my config for reference.

[general]
sync_module ptp ptp1
stats_log stderr
non_solarflare_nics on
timestamping_interfaces *

[ptp1]
ptp_mode master
ptp_tx_latency 0
ptp_rx_latency 0
ptp_bmc_priority1 1
ptp_bmc_priority2 1

[ptp]
interface bond0
priority 1
ptp_delay_mechanism end-to-end
ptp_network_mode hybrid
ptp_announce_interval -4
ptp_sync_pkt_interval -4
ptp_delayreq_interval -4
ptp_utc_offset 0
lacp_bypass sockpool

Before posting this issue, I did not have the timestamping_interfaces * line present. I discovered that upon inserting this line, sfptpd will consistently use two-step mode. I'm not sure why this is the case.

Anyway, I have another observation that sometimes upon starting the systemd unit, I will see the following log line:

info: interface: handling detected changes: eth1 (if_index 3)
info: ptp: interface bond0 timestamping changed hw -> sw
info: using SO_TIMESTAMPING_NEW software timestamps

Restarting the systemd unit seems to reliably resume using hw timestamps on the interface again. /var/lib/sfptpd/interfaces also reliably indicates that hw timestamping is available on this interface, so maybe this is an spurious issue from the driver or something? Any insight would be appreciated.

[abower-amd]

Hi @freeman94, thanks for getting in touch!

I'd like to separate your question out into two parts, the two-step clock mode and possibly incorrect timestamping mode detection in your bond. I will post separately on each.

Two-step clock

sfptpd does not itself send Sync or Pdelay_Resp messages as a one-step clock, and as you are using end-to-end delay mechanism only the former is relevant. Is sfptpd operating as a master clock in your use case [Re-reading your comment I see you are]?

• If you are then there is nothing to control in sfptpd because it will always act as a two-step clock when a master.
• If not then there is nothing to control in sfptpd as it will just work with whatever the other end is sending.

A possible source of confusion here is that the twoStepFlag flag is only set on the relevant PTP mesage. This can be confusing as we intiuitively expect to see it in the Announce message, where it is actually defined to be FALSE, and if it shows up as TRUE we cannot rely on that telling us anything. See IEEE Std 1588-2019 13.3.2.8 (Table 37).

Not relevant to you but if anyone wanted to add support for sfptpd as a one-step master then they are welcome, but we would not normally test that scenario as Solarflare NICs only offer two-step mode timestamping. There is no extra value to timestamping quality by using one-step mode and plenty of pitfalls! It could reduce network traffic, though.

[abower-amd]

@freeman94, part II:

Timestamping mode for bond

It would help if you could run with the following increased trace levels to capture what is happening in the case which is not working right for you:

trace_level general 4
trace_level netlink 4

and watch out for exactly what changes. Is there anything interesting in the bond apart from the physical timestamping NICs?

I notice that you are using the new LACP bond bypassing feature! This is designed for operating when the local ports within the bond are connected to different switches, to make sure the PTP routing is consistent. We didn't intend it for use against one switch although I guess that could potentially help eek out a little extra improvement if the paths within the switch might vary! Which is your topology

When the LACP bond bypass feature got added the logic for determining the timestamping mode for the bond changed. It's possible that that introduced a regression or that the logic was originally defective too. It could probably benefit from another review. We should probably be more dynamic and use whatever is relevant to the physical link used for the traffic, if we didn't already make that change with the bypass - I'd have to check. Cc. @jfeather-amd

[freeman94]

Andrew, thanks for your prompt reply.

If you are then there is nothing to control in sfptpd because it will always act as a two-step clock when a master.

Interesting, I believe I could produce a reproducible set of steps whereby sfptpd will only produce Sync messages but no Follow_Ups when it is configured for master-only operation. I'll try to get that to you tomorrow. Thanks for helping to explain the nuances to this concept.

Is there anything interesting in the bond apart from the physical timestamping NICs?

Yes, I think using the timestamping-interfaces * is my issue. Once I explicitly specify the interfaces with hw timestamping capability this issue appears to go away.

Interesting note about the LACP bypass feature. I didn't know whether or not it was necessary for me, just a bit of cruft leftover from testing as it doesn't seem to make an observable difference. I'll remove it for the time being.

[abower-amd]

If you are then there is nothing to control in sfptpd because it will always act as a two-step clock when a master.

Interesting, I believe I could produce a reproducible set of steps whereby sfptpd will only produce Sync messages but no Follow_Ups when it is configured for master-only operation. I'll try to get that to you tomorrow. Thanks for helping to explain the nuances to this concept.

Ah, I assumed you had been looking at the flag in the PTP packets. If your symptom is that the Follow_Up messages don't appear then that would happen if Tx timestamping is unsuccessful, which is interesting. In that case you would expect to see an alarm raised and messages in the log about failed transmission/timestamping.

Is there anything interesting in the bond apart from the physical timestamping NICs?

Yes, I think using the timestamping-interfaces * is my issue. Once I explicitly specify the interfaces with hw timestamping capability this issue appears to go away.

So everything works for you with this? I'm not satisfied by this resolution so if you can get some debug output from when it fails that would be super helpful, thanks!

[abower-amd]

Hi @freeman94, any news on your issue?