WIP

Author: ppalucki

deployment/pcm/README.md

@@ -7,13 +7,13 @@ Helm chart instructions
 - Configurable as non-privileged container (value: `privileged=false` / default) and privileged container,
 - Support for bare-metal and VM host configurations (files: [values-metal.yaml](values-metal.yaml), [values-vm.yaml](values-metal.yaml)),
 - Ability to deploy multiple releases alongside configured differently to handle different kinds of machines (bare-metal, VM) at the [same time](#heterogeneous-mixed-vmmetal-instances-cluster),
-- Controllable set of metrics and method of collection (RDT, uncore),  support direct (msr) and indirect (Linux abstractions perf/resctrl) counter accesses (file: `values-indirect.yaml`).
-- Linux Watchdog handling (controlled with PCM_KEEP_NMI_WATCHDOG, PCM_NO_AWS_WORKAROUND, nmiWatchdogMount values).
+- Controllable set of metrics and method of collection (RDT, uncore),  support direct (msr) and indirect (Linux abstractions perf/resctrl) counter accesses (file: [values-indirect.yaml](values-indirect.yaml)).
+- Linux Watchdog handling (controlled with `PCM_KEEP_NMI_WATCHDOG`, `PCM_NO_AWS_WORKAROUND`, `nmiWatchdogMount` values).
 - Deploy to own namespace with "helm install ... **-n pcm --create-namespace**"
 
 #### Integration features:
 
-- node-feature-discovery based nodeSelector and nodeAffinity (values: nfd, nfdBaremetalAffinity, nfdRDTAffinity),
+- node-feature-discovery based nodeSelector and nodeAffinity (values: `nfd`, `nfdBaremetalAffinity`, `nfdRDTAffinity`),
 - Examples for non-privileged mode using device plugin ("smarter-devices-manager") or using NRI device-injector plugin (TODO) (file: [values-smarter-devices-cpu-mem.yaml](values-smarter-devices-cpu-mem.yaml) ),
 - Integration with NRI balloons policy plugin (value: `nriBalloonsPolicyIntegration`),
 
@@ -41,21 +41,6 @@ helm upgrade --install pcm . --set privileged=true -f values-metal.yaml -f value
 helm install ... --set nfd=true --set podMonitor=true 
 ```
 
-#### DEBUGGING & BUILDING
-
-** NOTE: DEBUGGING: TODO to be remove before merging **
-
-```
-# Build local image for tests/development + fix /pcm/resctrl mounting (assuming project was configured with cmake previously):
-(cd ../.. ; (cd build ; make -j pcm pcm-sensor-server) ; docker build . -t localhost:5001/pcm-local && docker push localhost:5001/pcm-local; docker run -ti --rm --name pcmtest --entrypoint bash localhost:5001/pcm-local -c "pcm 2>&1 | head -5" )
-
-# Local image "indirect"
-helm upgrade --install pcm . --set debugPcm=true
-
-# exec or check logs
-kubectl exec -ti ds/pcm -- bash
-kubectl logs ds/pcm
-```
 
 ### Requirements
 
@@ -230,9 +215,12 @@ helm install pcm-metal . -f values-metal.yaml
 
 #### Direct method as non-privileged container (not recommended)
 
+**TODO**: TO BE MOVED TO EXTERNAL FILE/SECTION
+
 **Note** PCM requires access to /dev/cpu device in read writer mode (MSR access) but it is no possible currently to mount devices in Kubernetes pods/containers in vanila Kubernetes. Please read this isses for more information https://github.com/kubernetes/kubernetes/issues/5607.
 
 ##### a) Device injection using 3rd party device-plugin
+ 
 
 TO run PCM with as non privileged pod, we can third party devices plugins e.g.:
 
@@ -259,9 +247,9 @@ kubectl get node kind-control-plane -o json | jq .status.capacity
 helm install pcm . --set privileged=false -f values-direct.yaml -f values-smarter-devices-cpu-mem.yaml
 ```
 
-##### b) Device injection using NRI plugin device-injection
+##### b) Device injection using NRI plugin device-injection 
 
-**TODO**: **Warning** This is work in progress, because it is needed to manually specific all /dev/cpu/XX/msr devices, which is unpractical in production.
+**TODO**: **Warning** This is work in progress, because it is needed to manually specific all /dev/cpu/XX/msr devices, which is unpractical in production (TO BE MOVED TO EXTERNAL FILE).
 
 ```
 git clone https://github.com/containerd/nri/
@@ -294,7 +282,7 @@ docker exec kind-control-plane systemctl status device-injector
 helm install pcm-device-injector . --set privileged=false --set hostPort= --set debugSleep=true -f values-opcm-local-image.yaml -f values-device-injector.yaml
 ```
 
-#### Development (local images) and testing
+#### Development (with local images) and testing
 
 1) Setup kind with registry following this instruction: https://kind.sigs.k8s.io/docs/user/local-registry/
 ```
@@ -306,13 +294,33 @@ bash kind-with-registry.sh
 ```
 docker build . -t localhost:5001/pcm-local 
 docker push localhost:5001/pcm-local
+
+# or with single line
+# Build local image for tests/development + fix /pcm/resctrl mounting (assuming project was configured with cmake previously):
+(cd ../.. ; (cd build ; make -j pcm pcm-sensor-server) ; docker build . -t localhost:5001/pcm-local && docker push localhost:5001/pcm-local; docker run -ti --rm --name pcmtest --entrypoint bash localhost:5001/pcm-local -c "pcm 2>&1 | head -5" )
 ```
 
 3) When deploying to kind cluster pcm use values to switch to local pcm-local image
 ```
 helm install pcm . -f values-local-image.yaml
 ```
 
+4) Replace pcm-sensor-server with pcm or sleep
+```
+helm upgrade --install pcm . --set debugPcm=true
+helm upgrade --install pcm . --set debugSleep=true
+```
+
+**TODO:** consiert debug options to be removed before release for security reasons
+
+5) Check logs or intercat with container directly:
+```
+# exec into pcm container
+kubectl exec -ti ds/pcm -- bash
+# or check logs
+kubectl logs ds/pcm
+```
+
 #### Troubleshooting
 
 ##### Metric availability and requirements (devices/mounts/permissions)
@@ -347,10 +355,4 @@ helm install pcm . -f values-local-image.yaml
 |               | RO: /sys/firmware/acpi/tables/MCFG                         | PCM_USE_UNCORE_PERF             | msr is disabled       | pci.cpp:PciHandle::openMcfgTable()                       | mcfgMount                                           |
 |               | energy                                                     |                                 |                       | cpucounters.cpp initEnergyMonitoring()                   |                                                     |
 
-One can replace pcm-sensor-server command and run pcm or sleep to investigate issue add following arguments when install helm chart
-```
---debugPcm=true         # it will run pcm binary
---debugSleep=true       # will run "/usr/bin/sleep inf" to allow attach to container and debug interactivelly
-```
 
-**TODO:** options to be removed before release