gNB error: exceeded maximum number of timed out transmissions

[pchat-imm]

I have USRP B210. I run the open5gs docker as the core network. The gNB can connect but it always returns Error: exceeded maximum number of timed out transmissions. what did I do wrong? Thank you very much.

Below is my config files gnb_rf_b210_fdd_srsUE.yml. And how I run code (open5gs docker -> performance script -> gNB with config files). And i attacted my gnb log below

gnb_rf_b210_fdd_srsUE.yml

amf:
  addr: 10.53.1.2                                                 # The address or hostname of the AMF.
  bind_addr: 10.53.1.1                                            # A local IP that the gNB binds to for>

ru_sdr:
  device_driver: uhd                                              # The RF driver name.
  device_args: type=b200                                          # Optionally pass arguments to the selected RF driver.
  clock: internal
  sync:                                                           # (@!mm - changed from `external` to `internal`) Set sync to external. This set-up uses a LEO BODNAR GPDSO providing a 10 MHz ref.
  srate: 23.04                                                    # RF sample rate might need to be adjusted according to selected bandwidth.
  tx_gain: 89                                                     # Transmit gain of the RF might need to adjusted to the given situation.
  rx_gain: 76                                                     # Receive gain of the RF might need to adjusted to the given situation.

cell_cfg:
  dl_arfcn: 368500                                                # ARFCN of the downlink carrier (center frequency).
  band: 3                                                         # The NR band.
  channel_bandwidth_MHz: 20                                       # Bandwith in MHz. Number of PRBs will be automatically derived.
  common_scs: 15                                                  # Subcarrier spacing in kHz used for data.
  plmn: "00101"                                                   # PLMN broadcasted by the gNB.
  tac: 7                                                          # Tracking area code (needs to match the core configuration). 
  pdcch:
    dedicated:
      ss2_type: common                                            # Set search space to common to match capabilities
      dci_format_0_1_and_1_1: false                               # Use fallback DCI to match srsUE capabilities
    common:
      ss0_index: 0                                                # Set search space zero index to match srsUE capabilities
      coreset0_index: 12                                          # Set search CORESET Zero index to match srsUE capabilities
  prach:
    prach_config_index: 1                                         # Set PRACH config index to match srsUE expectation 

log:
  filename: /tmp/gnb.log                                          # Path of the log file.
  all_level: info                                                 # Logging level applied to all layers.

I run the open5gs docker

# run open5gs
>> cd srsRAN_Project/docker
>> sudo docker compose up --build 5gc

then run the performance script

# run performance script
>> echo "performance" | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
>> cd srsRAN_Project/scripts
>> ./srsran_performance 

after that i run the gnb code with config files

sudo ./gnb -c gnb_rf_b210_fdd_srsUE.yml 

The PRACH detector will not meet the performance requirements with the configuration {Format 0, ZCZ 0, SCS 1.25kHz, Rx ports 1}.
Lower PHY in quad executor mode.

--== srsRAN gNB (commit 2f90c8b60) ==--

Connecting to AMF on 10.53.1.2:38412
Available radio types: uhd and zmq.
[INFO] [UHD] linux; GNU C++ version 10.5.0; Boost_107400; UHD_4.0.0.HEAD-0-g90ce6062
[INFO] [LOGGING] Fastpath logging disabled at runtime.
Making USRP object with args 'type=b200'
[INFO] [B200] Detected Device: B210
[INFO] [B200] Operating over USB 3.
[INFO] [B200] Initialize CODEC control...
[INFO] [B200] Initialize Radio control...
[INFO] [B200] Performing register loopback test... 
[INFO] [B200] Register loopback test passed
[INFO] [B200] Performing register loopback test... 
[INFO] [B200] Register loopback test passed
[INFO] [B200] Setting master clock rate selection to 'automatic'.
[INFO] [B200] Asking for clock rate 16.000000 MHz... 
[INFO] [B200] Actually got clock rate 16.000000 MHz.
[INFO] [MULTI_USRP] Setting master clock rate selection to 'manual'.
[INFO] [B200] Asking for clock rate 23.040000 MHz... 
[INFO] [B200] Actually got clock rate 23.040000 MHz.
Cell pci=1, bw=20 MHz, dl_arfcn=368500 (n3), dl_freq=1842.5 MHz, dl_ssb_arfcn=368410, ul_freq=1747.5 MHz

==== gNodeB started ===
Type <t> to view trace
Late: 0; Underflow: 1; Overflow: 0;
Late: 0; Underflow: 1; Overflow: 0;
Late: 2796; Underflow: 1; Overflow: 1;
Error: exceeded maximum number of timed out transmissions.
Late: 6000; Underflow: 0; Overflow: 0;
Error: exceeded maximum number of timed out transmissions.
Late: 5988; Underflow: 2; Overflow: 0;
Error: exceeded maximum number of timed out transmissions.
Late: 6000; Underflow: 0; Overflow: 0;
Error: exceeded maximum number of timed out transmissions.

my gnb.log
gnb.log

[pgawlowicz]

is your PC powerful enough? Could you provide CPU and memory specs?

[pgawlowicz]

could you try to set the sync field in the config to internal? do you have an external clock to try?

[pchat-imm]

I don't have external clock to try.
I change the sync to internal, and it still have the same problem

ru_sdr:
  device_driver: uhd                                            
  device_args: type=b200                                          
  clock: internal
  sync: internal   

[pgawlowicz]

could you try to test the usrp with the UHD tools:
/usr/local/lib/uhd/examples/benchmark_rate --args type=b200 --rx_rate 23.04e6 --tx_rate 23.04e6
/usr/local/lib/uhd/examples/latency_test
/usr/local/lib/uhd/examples/test_timed_commands
/usr/local/lib/uhd/examples/tx_waveforms --freq=900e6 --rate=23.04e6

[pchat-imm]

all example tests is worked, I have attached results below.
I decided to make the sraRAN_Project again as I see the UHD version in gnb screen still said v4.0. After make, the version is said v4.1.0.

--== srsRAN gNB (commit 2f90c8b60) ==--

Connecting to AMF on 10.53.1.2:38412
Available radio types: uhd and zmq.
[INFO] [UHD] linux; GNU C++ version 10.5.0; Boost_107400; UHD_4.1.0.HEAD-0-gd21735d5

However, the problem still happended.
gnb.log

benchmark.txt
latency_test.txt
timeout.txt
tx_waveform.txt

[pgawlowicz]

hmm, ok, what is your system? could you check CPU load when runing gnb? with htop

[pchat-imm]

It will show some core has 100%. Does this means my computer is not sufficient to run the code? Is there any solution? Thank you

[pgawlowicz]

What OS do you use? do you run gnb in docker? Did you maybe play with expert_execution in the gnb config?

[pchat-imm]

I run on Ubuntu 22.04, and not gnb in docker. i install srsRAN_Project from source, and no expert_execution in gnb config

[pgawlowicz]

could you start with sudo taskset -c 0-7 ./gnb -c ./your_config_file

[pchat-imm]

I decide to erase and reinstall ubuntu 22.04 again. And install everything again, and it can works now! I can run the gNB for 10 mins right now, will test on longer time later.
Sometimes it still shows snapshot of htop as 100% in some one core. But the trace of gNB is working well, shows no problem.
Thank you very much

==== gNodeB started ===
Type <t> to view trace
Late: 19; Underflow: 9; Overflow: 0;
Late: 0; Underflow: 2; Overflow: 0;
Late: 10; Underflow: 3; Overflow: 0;
Late: 0; Underflow: 1; Overflow: 0;
Late: 0; Underflow: 1; Overflow: 0;
Late: 8; Underflow: 5; Overflow: 0;
Late: 0; Underflow: 1; Overflow: 0;
Late: 10; Underflow: 2; Overflow: 0;
Late: 5; Underflow: 1; Overflow: 0;
Late: 0; Underflow: 1; Overflow: 0;
Late: 0; Underflow: 5; Overflow: 0;
Late: 0; Underflow: 1; Overflow: 0;
Late: 0; Underflow: 1; Overflow: 0;
Late: 0; Underflow: 1; Overflow: 0;
Late: 5; Underflow: 1; Overflow: 0;
Late: 0; Underflow: 1; Overflow: 0;
Late: 0; Underflow: 2; Overflow: 0;
Late: 0; Underflow: 1; Overflow: 0;
Late: 0; Underflow: 4; Overflow: 0;
Late: 0; Underflow: 1; Overflow: 0;

[pgawlowicz]

Thanks for the feedback!

[houshangaziziaccelleran]

Hi
Is this issue already fixed? I have the same issue with my srsran/n310