USRP n310: exceeded maximum number of timed out transmissions.

[AzeezEbrahim]

Hey i am running USRP n310 and this error came and UEs doesn't connect.
Any help ?

Setup and connections:
Ubuntu 22.04
13th Gen Intel® Core™ i9-13900K × 32
128GB RAM
USRP n310 -> connected via SFP+ port to NIC card E810-XXVDA4T
(i am using UEs that does connect when using benetel RAN 550 or USRP b210)

config

amf:
  addr: 127.0.1.100                                             # The address or hostname of the AMF.
  bind_addr: 127.0.0.1                                          # A local IP that the gNB binds to for traffic from the AMF.

ru_sdr:
  device_driver: uhd                                            # The RF driver name.
  device_args: type=n3xx,addr=192.168.10.2  # Specify the clock source used by the RF.
  srate: 30.72                        # RF sample rate might need to be adjusted according to selected bandwidth.
  tx_gain: 45                         # Transmit gain of the RF might need to adjusted to the given situation.
  rx_gain: 30                         # Receive gain of the RF might need to adjusted to the given situation.

cell_cfg:
  dl_arfcn: 624628                                              # ARFCN of the downlink carrier (center frequency).
  band: 78                                                      # The NR band.
  channel_bandwidth_MHz: 20                                     # Bandwith in MHz. Number of PRBs will be automatically derived.
  common_scs: 30                                                # Subcarrier spacing in kHz used for data.
  plmn: "99970"                                                 # PLMN broadcasted by the gNB.
  tac: 7                                                        # Tracking area code (needs to match the core configuration).
  pci: 1                                                        # Physical cell ID.
  #nof_antennas_dl: 4
  #nof_antennas_ul: 4

log: 
 all_level: info                    # Optional TEXT (warning). Sets a common log level across PHY, MAC, RLC, PDCP, RRC, SDAP, NGAP and GTPU layers

log

024-01-17T19:59:07.581423 [GNB     ] [I] Radio Unit started successfully
2024-01-17T19:59:07.582848 [FAPI    ] [W] [    80.0] Real-time failure in FAPI: Received late UL_TTI.request from slot 79.19
2024-01-17T19:59:07.688888 [RF      ] [W] Real-time failure in RF: overflow
2024-01-17T19:59:07.690291 [RF      ] [W] Real-time failure in RF: underflow
2024-01-17T19:59:07.690594 [RF      ] [W] Real-time failure in RF: underflow
2024-01-17T19:59:07.690745 [RF      ] [W] Real-time failure in RF: underflow
2024-01-17T19:59:07.691048 [RF      ] [W] Real-time failure in RF: underflow
2024-01-17T19:59:07.691048 [RF      ] [W] Real-time failure in RF: underflow
2024-01-17T19:59:07.691200 [RF      ] [W] Real-time failure in RF: underflow
2024-01-17T19:59:07.691200 [RF      ] [W] Real-time failure in RF: underflow
2024-01-17T19:59:07.691352 [RF      ] [W] Real-time failure in RF: underflow
2024-01-17T19:59:07.691352 [RF      ] [W] Real-time failure in RF: underflow
2024-01-17T19:59:07.691352 [RF      ] [W] Real-time failure in RF: underflow
2024-01-17T19:59:07.691504 [RF      ] [W] Real-time failure in RF: underflow
2024-01-17T19:59:07.691504 [RF      ] [W] Real-time failure in RF: underflow
2024-01-17T19:59:07.691504 [RF      ] [W] Real-time failure in RF: underflow
2024-01-17T19:59:07.691656 [RF      ] [W] Real-time failure in RF: underflow
2024-01-17T19:59:07.691656 [RF      ] [W] Real-time failure in RF: underflow
2024-01-17T19:59:07.691656 [RF      ] [W] Real-time failure in RF: underflow
2024-01-17T19:59:07.691808 [RF      ] [W] Real-time failure in RF: underflow
2024-01-17T19:59:07.691808 [RF      ] [W] Real-time failure in RF: underflow
2024-01-17T19:59:07.691808 [RF      ] [W] Real-time failure in RF: underflow
2024-01-17T19:59:07.691960 [RF      ] [W] Real-time failure in RF: underflow
2024-01-17T19:59:07.691960 [RF      ] [W] Real-time failure in RF: underflow
...
2024-01-17T19:59:07.694348 [RF      ] [W] Real-time failure in RF: late
2024-01-17T19:59:07.694500 [RF      ] [W] Real-time failure in RF: late
2024-01-17T19:59:07.694500 [RF      ] [W] Real-time failure in RF: late
2024-01-17T19:59:07.694500 [RF      ] [W] Real-time failure in RF: late
2024-01-17T19:59:07.694652 [RF      ] [W] Real-time failure in RF: late
2024-01-17T19:59:07.694652 [RF      ] [W] Real-time failure in RF: late
..
...
2024-01-17T19:59:07.738683 [FAPI    ] [W] [    83.2] Real-time failure in FAPI: Received late DL_TTI.request from slot 83.0
2024-01-17T19:59:07.738687 [FAPI    ] [W] [    83.3] Real-time failure in FAPI: Received late DL_TTI.request from slot 83.1
2024-01-17T19:59:07.738691 [FAPI    ] [W] [    83.5] Real-time failure in FAPI: Received late DL_TTI.request from slot 83.2
2024-01-17T19:59:07.738695 [FAPI    ] [W] [    83.6] Real-time failure in FAPI: Received late DL_TTI.request from slot 83.3
2024-01-17T19:59:07.738698 [FAPI    ] [W] [    83.7] Real-time failure in FAPI: Received late DL_TTI.request from slot 83.4
2024-01-17T19:59:07.738701 [FAPI    ] [W] [    83.8] Real-time failure in FAPI: Received late DL_TTI.request from slot 83.5
2024-01-17T19:59:07.738704 [FAPI    ] [W] [    83.9] Real-time failure in FAPI: Received late DL_TTI.request from slot 83.6
2024-01-17T19:59:07.738707 [FAPI    ] [W] [    83.9] Real-time failure in FAPI: Received late UL_TTI.request from slot 83.7
2024-01-17T19:59:07.738709 [FAPI    ] [W] [   83.10] Real-time failure in FAPI: Received late UL_TTI.request from slot 83.8
2024-01-17T19:59:07.738711 [FAPI    ] [W] [   83.11] Real-time failure in FAPI: Received late UL_TTI.request from slot 83.9
2024-01-17T19:59:07.738714 [FAPI    ] [W] [   83.11] Real-time failure in FAPI: Received late DL_TTI.request from slot 83.10
2024-01-17T19:59:07.738717 [FAPI    ] [W] [   83.13] Real-time failure in FAPI: Received late DL_TTI.request from slot 83.11
2024-01-17T19:59:07.738721 [FAPI    ] [W] [   83.14] Real-time failure in FAPI: Received late DL_TTI.request from slot 83.12
2024-01-17T19:59:07.738724 [FAPI    ] [W] [   83.15] Real-time failure in FAPI: Received late DL_TTI.request from slot 83.13
2024-01-17T19:59:07.738731 [FAPI    ] [W] [   83.17] Real-time failure in FAPI: Received late DL_TTI.request from slot 83.14
2024-01-17T19:59:07.738735 [FAPI    ] [W] [   83.18] Real-time failure in FAPI: Received late DL_TTI.request from slot 83.15
2024-01-17T19:59:07.738738 [FAPI    ] [W] [   83.18] Real-time failure in FAPI: Received late DL_TTI.request from slot 83.16
2024-01-17T19:59:07.738740 [FAPI    ] [W] [   83.18] Real-time failure in FAPI: Received late UL_TTI.request from slot 83.17
2024-01-17T19:59:07.738744 [SCHED   ] [W] [   83.18] cell=0, slot=83.0: Discarding error indication. Cause: Scheduler results associated with the slot of the error indication have already been erased
2024-01-17T19:59:07.738744 [SCHED   ] [W] [   83.18] cell=0, slot=83.1: Discarding error indication. Cause: Scheduler results associated with the slot of the error indication have already been erased
2024-01-17T19:59:07.738744 [SCHED   ] [W] [   83.18] cell=0, slot=83.2: Discarding error indication. Cause: Scheduler results associated with the slot of the error indication have already been erased
2024-01-17T19:59:07.738744 [SCHED   ] [W] [   83.18] cell=0, slot=83.3: Discarding error indication. Cause: Scheduler results associated with the slot of the error indication have already been erased
2024-01-17T19:59:07.738744 [SCHED   ] [W] [   83.18] cell=0, slot=83.4: Discarding error indication. Cause: Scheduler results associated with the slot of the error indication have already been erased
2024-01-17T19:59:07.738744 [SCHED   ] [W] [   83.18] cell=0, slot=83.5: Discarding error indication. Cause: Scheduler results associated with the slot of the error indication have already been erased
2024-01-17T19:59:07.738744 [SCHED   ] [W] [   83.18] cell=0, slot=83.6: Discarding error indication. Cause: Scheduler results associated with the slot of the error indication have already been erased
2024-01-17T19:59:07.738744 [SCHED   ] [W] [   83.18] cell=0, slot=83.7: Discarding error indication. Cause: Scheduler results associated with the slot of the error indication have already been erased
2024-01-17T19:59:07.738744 [SCHED   ] [W] [   83.18] cell=0, slot=83.8: Discarding error indication. Cause: Scheduler results associated with the slot of the error indication have already been erased
2024-01-17T19:59:07.738744 [SCHED   ] [W] [   83.18] cell=0, slot=83.9: Discarding error indication. Cause: Scheduler results associated with the slot of the error indication have already been erased
2024-01-17T19:59:07.738744 [SCHED   ] [W] [   83.18] cell=0, slot=83.10: Discarding error indication. Cause: Scheduler results associated with the slot of the error indication have already been erased
2024-01-17T19:59:07.738745 [SCHED   ] [W] [   83.18] cell=0, slot=83.11: Discarding error indication. Cause: Scheduler results associated with the slot of the error indication have already been erased
2024-01-17T19:59:07.738745 [SCHED   ] [W] [   83.18] cell=0, slot=83.12: Discarding error indication. Cause: Scheduler results associated with the slot of the error indication have already been erased
2024-01-17T19:59:07.738745 [SCHED   ] [W] [   83.18] cell=0, slot=83.13: Discarding error indication. Cause: Scheduler results associated with the slot of the error indication have already been erased
2024-01-17T19:59:07.738745 [SCHED   ] [W] [   83.18] cell=0, slot=83.14: Discarding error indication. Cause: Scheduler results associated with the slot of the error indication have already been erased
2024-01-17T19:59:07.738844 [RF      ] [W] Real-time failure in RF: late

GNB

--== srsRAN gNB (commit dcd905cc6) ==--

Connecting to AMF on 127.0.1.100:38412
Available radio types: uhd and zmq.
[INFO] [UHD] linux; GNU C++ version 11.4.0; Boost_107400; UHD_4.6.0.0-0ubuntu1~jammy1
[INFO] [LOGGING] Fastpath logging disabled at runtime.
Making USRP object with args 'type=n3xx,addr=192.168.10.2,master_clock_rate=122.88e6,send_frame_size=8000,recv_frame_size=8000'
[INFO] [MPMD] Initializing 1 device(s) in parallel with args: mgmt_addr=192.168.10.2,type=n3xx,product=n310,serial=323F0CB,name=ni-n3xx-323F0CB,fpga=HG,claimed=False,addr=192.168.10.2,master_clock_rate=122.88e6,send_frame_size=8000,recv_frame_size=8000
[INFO] [MPM.PeriphManager] init() called with device args `fpga=HG,master_clock_rate=122.88e6,mgmt_addr=192.168.10.2,name=ni-n3xx-323F0CB,product=n310,recv_frame_size=8000,send_frame_size=8000,clock_source=internal,time_source=internal'.
[WARNING] [0/Radio#0] Attempting to set tick rate to 0. Skipping.
Cell pci=1, bw=20 MHz, dl_arfcn=624628 (n78), dl_freq=3369.42 MHz, dl_ssb_arfcn=624288, ul_freq=3369.42 MHz

==== gNodeB started ===
Type <t> to view trace
t
Error: failed receiving packet. RfnocError: OpTimeout: Control operation timed out waiting for ACK.
Late: 1573; Underflow: 583; Overflow: 6;
Error: exceeded maximum number of timed out transmissions.
Error: exceeded maximum number of timed out transmissions.
Error: exceeded maximum number of timed out transmissions.
Error: exceeded maximum number of timed out transmissions.
^CStopping ..

[AzeezEbrahim]

i discovered that i am connecting to port 0 SPF+ which support 1Gb only ( since i have HG version).
so i just changed to port 1 SPF+ (10Gb).

[shench17]

Hello, I’ve encountered the same issue as you did when I was running gNB. I noticed that you solve it by switching from SPF+ port 0 to SPF+ port 1. Could you kindly explain how you made this port switch? Could you share the steps you took? Your help would be greatly appreciated. Thank you!

[pgawlowicz]

do you have an SFP port that supports 10Gbps?

[shench17]

do you have an SFP port that supports 10Gbps?

Yes, the device I used was the USRP X310, which has an SFP port that supports 10Gbps.

[pgawlowicz]

You might try to add subdev='B:0 B:0'" to ru_sdr.device_args in gnb config file. However, I am not sure if this will be handled by the code.