V23.4 external release (#67)

* v23.4 external release

Author: Tangrui333

CMakeLists.txt

@@ -123,6 +123,7 @@ if(BACKEND)
     include(${BACKEND} OPTIONAL)
 endif()
 set(IMAGEARCH "linux/amd64")
+set(IMAGESUFFIX "")
 include(multi-arch OPTIONAL)
 
 foreach(subdir image stack workload dist)

doc/user-guide/preparing-infrastructure/setup-containerd.md

@@ -45,6 +45,18 @@ EOF
 sudo systemctl restart containerd
 ```
 
+On-Premises workload validation based on RKE2 requires to use a docker registry. If you need to setup any insecure registries with `containerd`, modify the `rke2` configuration as follows, assuming your private registry is `foo.com:5000`:   
+
+```shell
+cat | sudo tee /etc/rancher/rke2/registries.yaml <<EOF
+mirrors:
+  "foo.com:5000":
+    endpoint:
+      - "http://foo.com:5000"
+EOF 
+sudo systemctl restart rke2-server
+```
+
 ## Setup Data Storage
 
 Optionally, if you need to move the containerd storage location to, for example, `/mnt/storage/containerd`:

doc/user-guide/preparing-infrastructure/setup-dlb.md

@@ -0,0 +1,24 @@
+
+# DLB Setup
+
+DLB is supported in certain SPR SKUs. Please make sure your CPU sku (QDF) supports DLB.
+
+And DLB is broken with latest BKC kernel `5.15.0-spr.bkc.pc.2.10.0.x86_64`.
+Before running the workload, please make sure your SPR has DLB device by running the following command:
+
+```shell
+lspci | grep 2710
+```
+
+If there are devices listed, then please download the DLB driver from this link: https://www.intel.com/content/www/us/en/download/686372/intel-dynamic-load-balancer.html
+
+Execute the following commands:
+
+```shell
+tar -xf dlb_linux_src_release_<dlb_driver_version>.txz
+cd dlb/driver/dlb2/
+make
+sudo insmod dlb2.ko
+```
+
+Then you can run the workload on this machine.

doc/user-guide/preparing-infrastructure/setup-nginx-cache.md

@@ -0,0 +1,65 @@
+# Nginx Cache Server Setup
+
+This document is a guide for setting up Nginx-DLB benchmark environment, including Hardware platform and Software configuration on storage.
+
+## HW Prerequisites
+
+- Setup 3 servers: one node (client) for client deployment; 2 node for cache server (worker-0) and content server (worker-1) deployment.
+- Cache server (worker-0) requires 5 NVMe disks.
+
+## K8S Labels configuaration
+
+Please finish the section, [Storage configuration](setup-nginx-cache.md#storage-configuration), then label the corresponding nodes.
+
+Command examples:
+
+- Label:
+  ```shell
+  kubectl label node <node name> HAS-SETUP-NVMECACHE=yes
+  ```
+- Unlabel:
+  ```shell
+  kubectl label node <node name> HAS-SETUP-NVMECACHE-
+  ```
+
+Nginx cache server worker-0:*
+
+- `HAS-SETUP-NGINX-CACHE=yes`
+
+## Storage configuration
+
+This should be done on worker-0.
+
+- Prepare 5 nvme disk for nginx cache server pod. *nvme?* means multiple nvme disk.
+
+  - Check NVMe drives and Partition drives
+    ```shell command
+    ls /dev/nvme*
+    ```
+
+    ```output
+    /dev/nvme1    /dev/nvme2    /dev/nvme3    /dev/nvme4    /dev/nvme5
+    ```
+
+  - Format drives as ext4 (or xfs):
+    ```shell command
+    mkfs.ext4 /dev/nvme1n1
+    mkfs.ext4 /dev/nvme2n1
+    mkfs.ext4 /dev/nvme3n1
+    mkfs.ext4 /dev/nvme4n1
+    mkfs.ext4 /dev/nvme5n1
+    ```
+
+  - Create cache mountpoints and mount to four pairs
+    ```shell command
+    mkdir /nginx/cache1
+    mount -o rw,noatime,seclabel,discard /dev/nvme1n1 /nginx/cache1
+    mkdir /nginx/cache2
+    mount -o rw,noatime,seclabel,discard /dev/nvme2n1 /nginx/cache2
+    mkdir /nginx/cache3
+    mount -o rw,noatime,seclabel,discard /dev/nvme3n1 /nginx/cache3
+    mkdir /nginx/cache4
+    mount -o rw,noatime,seclabel,discard /dev/nvme4n1 /nginx/cache4
+    mkdir /nginx/cache5
+    mount -o rw,noatime,seclabel,discard /dev/nvme5n1 /nginx/cache5
+    ```

doc/user-guide/preparing-infrastructure/setup-terraform.md

@@ -180,5 +180,4 @@ $ exit
 [WSF Cloud Setup]: setup-wsf.md#cloud-development-setup
 [WSF On-Premises Setup]: setup-wsf.md#on-premises-development-setup
 [terraform-config.static.tf]: ../../../script/terraform/terraform-config.static.tf
-[WSF KVM Setup]: setup-wsf.md#kvm-development-setup
 [Trace Module]: ../executing-workload/terraform-options.md#trace-module-parameters

doc/user-guide/preparing-infrastructure/setup-wsf.md

@@ -19,7 +19,7 @@ flowchart TD;
   net <--> sut3;;
 ```
 
-where the SUT machines can be physical hosts in the On-Premesis case, or virtualized VMs in the Cloud or KVM execution. If Kubernetes is used, it is assumed that the SUT hosts form a Kubernetes cluster, where one of the SUTs be the Kubernetes controller.  
+where the SUT machines can be physical hosts in the On-Premesis case, or virtualized VMs in the Cloud. If Kubernetes is used, it is assumed that the SUT hosts form a Kubernetes cluster, where one of the SUTs be the Kubernetes controller.  
 
 There can be many variations of the above diagram:
 - Combine dev and the Kubernetes controller on the same machine.  
@@ -167,18 +167,16 @@ Use the following setup steps:
 
 - Use `cmake -DREGISTRY=<registry_url> ..` to set the private registry URL.
 - Set `k8s_enable_registry: false` in `script/terraform/terraform-config.static.tf`. This is the default.
-- If the host has enough CPU cores and memory, suggest to use the KVM development setup for more flexibility.
 
 ---
 
 ## Setup Scripts
-> Note: All scripts mentioned below, such as setup-dev.sh, setup-sut-kvm.sh, etc, are found inside ```script/setup/``` and must be executed from there.
+> Note: All scripts mentioned below, such as setup-dev.sh, etc, are found inside ```script/setup/``` and must be executed from there.
 - **[`setup-dev.sh`][setup-dev.sh-self]**: Setup the dev host.   
 - **[`setup-reg.sh`][setup-reg.sh-self]**: Setup a private docker registry.   
 - **[`setup-sut-native.sh`][setup-sut-native.sh-self]**: Setup the SUT host for native workload execution.  
 - **[`setup-sut-docker.sh`][setup-sut-docker.sh-self]**: Setup the SUT host for docker/docker-compose workload execution.  
 - **[`setup-sut-k8s.sh`][setup-sut-k8s.sh-self]**: Setup the SUT host for Kubernetes workload execution.  
-- **[`setup-sut-kvm.sh`][setup-sut-kvm.sh-self]**: Setup the KVM host.  
 
 ### setup-dev.sh
 
@@ -266,7 +264,6 @@ where `options` are:
 
 [Instructions of Cloud Setup]: #cloud-setup
 [Instructions of On-Premises Setup]: #on-premises-setup
-[Instructions of KVM Setup]: #kvm-setup
 [Manual of Setup Scripts]: #setup-scripts
 [Prerequisites]: #prerequisites
 [RFC-1178]: http://www.faqs.org/rfcs/rfc1178.html

script/benchmark/ctest.sh

@@ -121,7 +121,7 @@ if [ -n "$stop" ]; then
     if [ "$($cmd | wc -l)" -ne 2 ] && [ "$stop" != "--stop=all" ]; then
         echo "None or multiple ctest instances detected:"
         echo ""
-        $cmd --format '{{.Names}}\t\t{{.ID}}\t{{.Status}}'
+        $cmd --format '{{.Names}}\t{{.Status}}'
         echo ""
         echo "Please identify the instance with: ./ctest.sh --stop=<prefix> or ./ctest.sh --stop=all"
         exit 3
@@ -255,7 +255,7 @@ for var in "$@"; do
     --check-docker-image)
         export CTESTSH_OPTIONS="$CTESTSH_OPTIONS --check-docker-image"
         ;;
-    --push-docker-image=)
+    --push-docker-image=*)
         export CTESTSH_OPTIONS="$CTESTSH_OPTIONS --push-docker-image=${var#--push-docker-image=}"
         ;;
     --push-docker-image)

script/benchmark/kpi-list.awk

@@ -5,6 +5,9 @@
 # SPDX-License-Identifier: Apache-2.0
 #
 
+BEGIN {
+    status="failed"
+}
 /^#svrinfo[:-] / {
     if (!svrinfo) next
 }
@@ -14,7 +17,10 @@
 {   
     print $0
 }
-/^[*].*: *([0-9.-][0-9.e+-]*) *#*.*$/ {
+/^# status: (passed|failed)/ {
+    status=$3
+}
+/^[*].*: *([0-9.-][0-9.e+-]*) *#*.*$/ && status=="passed" {
     k=gensub(/^(.*): *[0-9.-][0-9.-]*.*$/,"\\1",1,$0)
     v=gensub(/^.*: *([0-9.-][0-9.-]*).*$/,"\\1",1,$0)
     n[k]=n[k]+1

script/benchmark/kpi-xls-ai.awk

@@ -26,6 +26,11 @@ function get_value() {
 /\/itr-[0-9]*:$/{
     name=gensub(/^.*logs-([^/]*)[/].*$/,"\\1",1)
     itr=gensub(/^.*[/]itr-([0-9]+):$/,"\\1",1)
+    status="failed"
+}
+
+/^# status: (passed|failed)/ {
+    status=$3
 }
 
 index($0,var1)==1 || ($1=="#" && index($2,var1)==1) {
@@ -36,14 +41,14 @@ index($0,var2)==1 || ($1=="#" && index($2,var2)==1) {
     var2v=gensub(/.*"(.*)".*/,"\\1",1,$NF)
 }
 
-index($0,var3)==1 {
+index($0,var3)==1 && status=="passed" {
     var3v[name][product][var1v][var2v][++var3vct[name][product][var1v][var2v]]=get_value()
     n=length(var3v[name][product][var1v][var2v])
     if (n>var34n[name][product][var1v][var2v])
         var34n[name][product][var1v][var2v]=n
 }
 
-index($0,var4)==1 {
+index($0,var4)==1 && status=="passed" {
     idx=gensub(/ *([0-9]+).*$/,"\\1",1,substr($0,length(var4)+1))
     var4v[name][product][var1v][var2v][idx][++var4vct[name][product][var1v][var2v][idx]]=get_value()
     n=length(var4v[name][product][var1v][var2v][idx])

script/benchmark/kpi-xls-inst.awk

@@ -12,9 +12,14 @@
 
 /\/itr-[0-9]*:$/ {
     name=gensub(/^.*logs-([^/]*)[/].*$/,"\\1",1)
+    status="failed"
 }
 
-/^[^#].*: *[0-9.-][0-9.e+-]* *#?.*$/ {
+/^# status: (passed|failed)/ {
+    status=$3
+}
+
+/^[^#].*: *[0-9.-][0-9.e+-]* *#?.*$/ && status=="passed" {
     k=gensub(/^(.*): *[0-9.-]+.*$/, "\\1", 1)
     v=gensub(/^.*: *([0-9.-]+).*/, "\\1", 1)
     kpis[name][product][k][++kpisct[name][product][k]]=v
@@ -90,12 +95,14 @@ END {
             print "</Row>"
         }
 
-        add_svrinfo_ex(ws, psp, ith)
+        if (length(svrinfo_values[ws])>0)
+            add_svrinfo_ex(ws, psp, ith)
 
         print "</Table>"
         print "</Worksheet>"
 
-        add_svrinfo(ws)
+        if (length(svrinfo_values[ws])>0)
+            add_svrinfo(ws)
     }
     print "</Workbook>"
 }