CPU Pinning and optimization

[IbrahimBabiker87]

Hello SrsRAN,

I have a machine with 32 vcpu ( Hyper threading), I'm isolating vcpu from 4~31 dedicated for Srsran, I want to make sure that srsran will make a maximum use of the available cpus, I used below settings,
However what is the difference between those policies :

1. round-robin
2. mask

how I assign a set of cpus to specific function for example l1_dl_cpus cpu_set == [2,3,4,5,6,7,8]
and is there a maximum limit for cpu assignment per function and is it different from function to other for instance l1_ul will be using a maximum of 4 cpus.

expert_execution:
 
 affinities: 
     l1_dl_cpus: 23                                # Optional TEXT. Sets the number of CPU cores assigned to L1 downlink tasks. Supported: [1 - Number of available cores].
     l1_ul_cpus: 23                              # Optional TEXT. Sets the number of CPU cores assigned to L1 uplink tasks. Supported: [1 - Number of available cores].
     l2_cell_cpus: 23                              # Optional TEXT. Sets the number of CPU cores assigned to L2 cells tasks. Supported: [1 - Number of available cores].
     low_priority_cpus: 23                          # Optional TEXT. Sets the number of CPU cores assigned to low priority tasks. Supported: [1 - Number of available cores].
#    ru_cpus: 5                                    # Optional TEXT. Sets the number of CPU cores used for the Radio Unit tasks. Supported: [1 - Number of available cores].
     l1_dl_pinning: round-robin                            # Optional TEXT. Sets the policy used for assigning CPU cores to L1 downlink tasks.
     l1_ul_pinning: round-robin                              # Optional TEXT. Sets the policy used for assigning CPU cores to L1 uplink tasks. 
     l2_cell_pinning: round-robin                             # Optional TEXT. Sets the policy used for assigning CPU cores to L2 cell tasks.
     low_priority_pinning: round-robin                        # Optional TEXT. Sets the policy used for assigning CPU cores to low priority tasks.
#    ru_pinning:                                 # Optional TEXT. Sets the policy used for assigning CPU cores to Radio Unity tasks.

 threads: 
    upper_phy: 
     pdsch_processor_type: auto                    # Optional TEXT (auto). Sets the PDSCH processor type. Supported: [auto, generic, concurrent, lite].
     nof_pdsch_threads: 4                          # Optional UINT (4). Sets the number of threads used to encode PDSCH.
#      nof_pusch_threads: 1                          # Optional UINT (1). Sets the number of threads used to encode PUSCH.
     nof_ul_threads: 7                           # Optional UINT (1). Sets the number of upprt PHY threads to proccess uplink.
     nof_dl_threads: 4                             # Optional UINT (1). Sets the number of upprt PHY threads to proccess downlink.  

[ismagom]

Hi @IbrahimBabiker87 ,

You need to define a set of core indexes in that affinity option, for instance:

affinities:
 l1_dl_cpus: 2,3,4,5,6,7,8
 l1_ul_cpus: 2-8
 ru_cpus: 2-4,8-10

If you set 23 as you configured, it will pin all threads to core index 23 only, which is probably not what you want.

The difference between the policies is:

• round robin: If there are 5 total DL threads in PHY DL/PDSCH and you configured l1_dl_cpus as my example above, it will pin each thread as follows: 1st thread to core 2, 2nd thread to core 3, 3rd thread to core 4, etc.
• mask: it will pin the 5 DL threads to all set, allowing them to migrate from one core to another.

In our experience the policy mask gives better results (it's the default).

One more comment, the option nof_pusch_threads should be replaced by nof_pusch_decoder_threads.

[IbrahimBabiker87]

I do have 32 cpus, but the srsGNB not accepting larger than 24 (23)

srsGNB ERROR: Invalid CPU core selected '31'. Valid range is [0-23] in the 'low_priority_cpus' property
Aborted (core dumped)

[ismagom]

Are you maybe isolating some cores? What does lscpu -p return? The maximum cores we accept is obtained from the OS through the std::thread::hardware_concurrency() function. So it seems the OS is only seeing 24 cores.

[IbrahimBabiker87]

Yes, Im isolating 24 cores, So the indexing will be considering the taskset passed to SRSRAN or not ?
for instance the isolated cores as isolcpus=4-15,20-31 " trying to keep vcpu pairs " siblings" together"

sudo taskset -c 4,5,6,7,8,9,10,11,12,13,14,15,20,21,22,23,24,25,26,27,28,29,30,31 gnb -c N310_100MHz.yml
for srsRAN indexing is belowmapping is correct ??

4 == 0
5 == 1
6 == 2
7 == 3
8 == 4
.
.
.
31 == 23

[ismagom]

Yes, srsRAN sees only the cores passed by taskset