Merge pull request #205 from sfeifer/main

Multiple changes including transition from PCP Redis to PCP Valkey

Author: sfeifer

.github/ISSUE_TEMPLATE/bug_report.md

@@ -29,7 +29,7 @@ If applicable, add screenshots to help explain your problem.
  - Performance Co-Pilot:
  - Grafana:
  - grafana-pcp:
-- Redis (if applicable):
+- Valkey (if applicable):
 
 **Additional context**
 Add any other context about the problem here.

.github/workflows/ci.yml

@@ -72,7 +72,7 @@ jobs:
           name: build
           path: build
 
-      - name: Start PCP and Redis in a pod
+      - name: Start PCP and Valkey in a pod
         run: |
           # workaround required on GitHub Actions to allow running systemd in rootless containers
           # https://github.com/actions/virtual-environments/issues/3536

CHANGELOG.md

@@ -1,8 +1,20 @@
 # Change Log
 
-## 5.1.2 (unreleased)
-
+## 5.2.0 (2024-11-08)
+
+* **valkey**: new PCP Valkey datasource (replacement for PCP Redis)
+* **redis**: PCP Redis datasource renamed to PCP Valkey due to licensing
+* **dashboards**: new PCP Vector Top Consumers dashboard
+* **dashboards**: new PCP Vector UWSGI overview dashboard
+* **dashboards**: new PCP Prometheus Host Overview dashboard
+* **test**: update cypress tests for grafana 10
+* **docs**: update donations page link with new fiscal sponsor details
+* **docs**: add .readthedocs.yaml file to keep grafana-pcp.readthedocs.io working
+* **docs**: fix command in installation doc to allow loading unsigned plugins for both containers and from source
 * **build**: update grafana/plugin-validator and pin nodejs version
+* **build**: update Go and node dependencies
+* **ci**: update release scrip to use gh instead of deprecated hub command
+* **search**: metric search turned off until Valkey replacement arrives for RediSearch
 
 
 ## 5.1.1 (2022-10-27)

Makefile

@@ -36,11 +36,11 @@ watch-frontend: deps-frontend build-dashboards ## Auto rebuilt frontend on file
 	yarn run watch
 
 dev-backend: deps-backend
-	go build -race -o ./dist/datasources/redis/pcp_redis_datasource_$$(go env GOOS)_$$(go env GOARCH) -tags netgo -ldflags -w ./pkg
+	go build -race -o ./dist/datasources/valkey/pcp_valkey_datasource_$$(go env GOOS)_$$(go env GOARCH) -tags netgo -ldflags -w ./pkg
 
 restart-backend: ## Rebuild and restart backend data source (as root)
 	sudo -u "$$(stat -c '%U' .)" make dev-backend
-	killall pcp_redis_datasource_$$(go env GOOS)_$$(go env GOARCH)
+	killall pcp_valkey_datasource_$$(go env GOOS)_$$(go env GOARCH)
 
 
 ##@ Build
@@ -61,10 +61,10 @@ GO_LD_FLAGS := -w -s -extldflags "-static"
 build-backend: deps-backend ## Build backend data source
 	#mage buildAll
 	for arch in amd64 arm arm64 s390x ppc64le 386; do \
-	  CGO_ENABLED=0 GOOS=linux GOARCH=$${arch} go build -o dist/datasources/redis/pcp_redis_datasource_linux_$${arch} -ldflags '$(GO_LD_FLAGS)' ./pkg; \
+	  CGO_ENABLED=0 GOOS=linux GOARCH=$${arch} go build -o dist/datasources/valkey/pcp_valkey_datasource_linux_$${arch} -ldflags '$(GO_LD_FLAGS)' ./pkg; \
 	done
-	CGO_ENABLED=0 GOOS=darwin GOARCH=amd64 go build -o dist/datasources/redis/pcp_redis_datasource_darwin_amd64 -ldflags '$(GO_LD_FLAGS)' ./pkg
-	CGO_ENABLED=0 GOOS=windows GOARCH=amd64 go build -o dist/datasources/redis/pcp_redis_datasource_windows_amd64.exe -ldflags '$(GO_LD_FLAGS)' ./pkg
+	CGO_ENABLED=0 GOOS=darwin GOARCH=amd64 go build -o dist/datasources/valkey/pcp_valkey_datasource_darwin_amd64 -ldflags '$(GO_LD_FLAGS)' ./pkg
+	CGO_ENABLED=0 GOOS=windows GOARCH=amd64 go build -o dist/datasources/valkey/pcp_valkey_datasource_windows_amd64.exe -ldflags '$(GO_LD_FLAGS)' ./pkg
 
 build: build-dashboards build-frontend build-backend ## Build everything
 
@@ -99,21 +99,21 @@ test: test-frontend test-backend ## Run all tests
 
 ##@ UI tests
 
-test-ui-start-pod: ## Start PCP and Redis in a pod
+test-ui-start-pod: ## Start PCP and Valkey in a pod
 	-podman pod rm -f grafana-pcp-tests
 	podman pod create --name grafana-pcp-tests -p 3001:3000
 	podman run --pod grafana-pcp-tests --name grafana-pcp-tests-pcp -d --systemd always quay.io/performancecopilot/pcp
-	podman run --pod grafana-pcp-tests --name grafana-pcp-tests-redis -d docker.io/library/redis:6
+	podman run --pod grafana-pcp-tests --name grafana-pcp-tests-valkey -d docker.io/valkey/valkey:7
 
 test-ui-start-grafana-dist: ## Start Grafana with grafana-pcp from the dist/ folder
 	podman run --pod grafana-pcp-tests --name grafana-pcp-tests-grafana -d --replace \
-		-e GF_PLUGINS_ALLOW_LOADING_UNSIGNED_PLUGINS="performancecopilot-pcp-app,performancecopilot-redis-datasource,performancecopilot-vector-datasource,performancecopilot-bpftrace-datasource,performancecopilot-flamegraph-panel,performancecopilot-breadcrumbs-panel,performancecopilot-troubleshooting-panel" \
+		-e GF_PLUGINS_ALLOW_LOADING_UNSIGNED_PLUGINS="performancecopilot-pcp-app,performancecopilot-valkey-datasource,performancecopilot-vector-datasource,performancecopilot-bpftrace-datasource,performancecopilot-flamegraph-panel,performancecopilot-breadcrumbs-panel,performancecopilot-troubleshooting-panel" \
 		-v $$(pwd)/dist:/var/lib/grafana/plugins/performancecopilot-pcp-app \
 		$(GRAFANA_IMAGE)
 
 test-ui-start-grafana-build: ## Start Grafana with grafana-pcp from build/performancecopilot-pcp-app-*.zip
 	podman run --pod grafana-pcp-tests --name grafana-pcp-tests-grafana -d --replace \
-		-e GF_PLUGINS_ALLOW_LOADING_UNSIGNED_PLUGINS="performancecopilot-pcp-app,performancecopilot-redis-datasource,performancecopilot-vector-datasource,performancecopilot-bpftrace-datasource,performancecopilot-flamegraph-panel,performancecopilot-breadcrumbs-panel,performancecopilot-troubleshooting-panel" \
+		-e GF_PLUGINS_ALLOW_LOADING_UNSIGNED_PLUGINS="performancecopilot-pcp-app,performancecopilot-valkey-datasource,performancecopilot-vector-datasource,performancecopilot-bpftrace-datasource,performancecopilot-flamegraph-panel,performancecopilot-breadcrumbs-panel,performancecopilot-troubleshooting-panel" \
 		-e GF_INSTALL_PLUGINS="/tmp/plugin.zip;performancecopilot-pcp-app" \
 		-v $$(pwd)/build/$$(basename build/performancecopilot-pcp-app-*.zip):/tmp/plugin.zip \
 		$(GRAFANA_IMAGE)

README.md

@@ -31,7 +31,7 @@ It presents a unifying abstraction for all of the performance data in a system,
 * support for repeated panels
 * sample dashboards for all data sources
 
-[1] PCP Redis
+[1] PCP Valkey
 [2] PCP Vector
 [3] PCP bpftrace
 

ci/scripts/github-release.sh

@@ -16,10 +16,10 @@ if [[ "${GRAFANA_PLUGIN_VERSION}" == *beta* ]]; then
   PRERELEASE_ARG="-p"
 fi
 
-hub release create \
-    -m "grafana-pcp v${GRAFANA_PLUGIN_VERSION}" \
-    -m "${RELEASE_NOTES}" \
-    -a "build/${GRAFANA_PLUGIN_ARTIFACT}" \
-    -a "build/${GRAFANA_PLUGIN_ARTIFACT_CHECKSUM}" \
-    $PRERELEASE_ARG \
-    "v${GRAFANA_PLUGIN_VERSION}"
+gh release create \
+    "v${GRAFANA_PLUGIN_VERSION}" \
+    -t "grafana-pcp v${GRAFANA_PLUGIN_VERSION}" \
+    -n "${RELEASE_NOTES}" \
+    "build/${GRAFANA_PLUGIN_ARTIFACT}" \
+    "build/${GRAFANA_PLUGIN_ARTIFACT_CHECKSUM}" \
+    ${PRERELEASE_ARG}

cypress/e2e/datasources/redis.cy.ts cypress/e2e/datasources/valkey.cy.tscypress/e2e/datasources/redis.cy.ts renamed to cypress/e2e/datasources/valkey.cy.ts

@@ -1,9 +1,9 @@
-describe('PCP Redis data source', () => {
+describe('PCP Valkey data source', () => {
     before(() => {
         cy.task('grafana:reset');
         cy.login();
         cy.enablePlugin();
-        cy.addDatasource('performancecopilot-redis-datasource', 'PCP Redis data source');
+        cy.addDatasource('performancecopilot-valkey-datasource', 'PCP Valkey data source');
     });
 
     beforeEach(() => {
@@ -12,9 +12,9 @@ describe('PCP Redis data source', () => {
 
     it('should import bundled dashboards', () => {
         cy.visit('/datasources');
-        cy.contains('PCP Redis').click();
+        cy.contains('PCP Valkey').click();
         cy.contains('Dashboards').click({ force: true });
-        cy.contains('td', 'PCP Redis: Host Overview').siblings().contains('Import').click({ force: true });
+        cy.contains('td', 'PCP Valkey: Host Overview').siblings().contains('Import').click({ force: true });
         cy.contains('Dashboard Imported');
     });
 
@@ -27,7 +27,7 @@ describe('PCP Redis data source', () => {
               cy.contains('Add a new panel').click();
             } else {
               cy.contains('Add visualization').click();
-              cy.get('div.scrollbar-view').contains('PCP Redis').click();
+              cy.get('div.scrollbar-view').contains('PCP Valkey').click();
             }
         });
 

cypress/e2e/setup.cy.ts

@@ -13,9 +13,9 @@ describe('grafana-pcp setup', () => {
         cy.get('button').should('include.text', 'Disable');
     });
 
-    it('should setup PCP Redis data source', () => {
+    it('should setup PCP Valkey data source', () => {
         cy.visit('/datasources/new');
-        cy.contains('PCP Redis').click();
+        cy.contains('PCP Valkey').click();
         cy.get('input[placeholder="http://localhost:44322"]').type('http://localhost:44322');
         cy.get('button[type=submit]').click();
         cy.contains('Data source is working');

docs/architecture.rst

@@ -14,18 +14,18 @@ Monitored hosts run the **Performance Metrics Collector Daemon (PMCD)**, which c
 Each **PMDA** is responsible for gathering metrics of one specific domain - e.g., the kernel, services (e.g., PostgreSQL), or other instrumented applications.
 The **pmlogger** daemon records metrics from **pmcd** and stores them in archive files on the hard drive.
 
-Since **PCP 5** metrics can also be stored in the redis database, which allows multi-host performance analysis, the **pmproxy** daemon discovers new archives (created by **pmlogger**) and stores them in a redis database.
+Since **PCP 5** metrics can also be stored in the valkey database, which allows multi-host performance analysis, the **pmproxy** daemon discovers new archives (created by **pmlogger**) and stores them in a valkey database.
 
 Dashboards
 ----------
 
 Performance Co-Pilot metrics can be analyzed with Grafana dashboards, using the **grafana-pcp** plugin.
 There are two modes available:
 
-* historical metrics across multiple hosts using the :doc:`datasources/redis` data source
+* historical metrics across multiple hosts using the :doc:`datasources/valkey` data source
 * live, on-host metrics using the :doc:`datasources/vector` data source
 
-The :doc:`datasources/redis` data source sends :ref:`pmseries <pmseries-query-language>` queries to **pmproxy**, which in turn queries the redis database for metrics.
+The :doc:`datasources/valkey` data source sends :ref:`pmseries <pmseries-query-language>` queries to **pmproxy**, which in turn queries the valkey database for metrics.
 The :doc:`datasources/vector` data source connects to **pmproxy**, which in turn requests live metrics directly from a local or remote **PMCD**.
 In this case, metrics are stored temporarily in the browser, and metric values are lost when the browser tab is refreshed.
-The :doc:`PCP Redis data source <datasources/redis>` is required for persistence.
+The :doc:`PCP Valkey data source <datasources/valkey>` is required for persistence.

docs/datasources/index.rst

@@ -3,7 +3,7 @@ Overview
 
 .. include:: ../refs.rst
 
-PCP Redis
+PCP Valkey
 ---------
 
 This data source queries the fast, scalable time series capabilities provided by the `pmseries`_ functionality.