bootlinux: add support for custom refs on dev kernels on the CLI

We already have support to easily test arbitrary kernels and refs by
just using the LINUX_TREE and LINUX_REF environment variables. We
leverage this for our CIs:

  * LINUX_TREE: the target tree     for baseline group (A)
  * LINUX_REF:  the target tree ref the baseline group (A)

Now that we have AB-testing support we want to first add support so that
a simple TEST_AB=y will enable KDEVOPS_BASELINE_AND_DEV and then we want
to be able to customize the target dev tree and ref. We do this with two
other environment variables:

  * LINUX_DEV_TREE: the target tree     for dev group (B)
  * LINUX_DEV_REF:  the target tree ref the dev group (B)

You just need to make sure to also pass TEST_AB=y.

By leveraging this we can easily use AB testing in the future on CIs
to compare and contrast different kernel releases against any target
workload we have.

For example, here's the power of how to use this, suppose we want to
test the branch "20250723-migrate-folio-stats" on my kernel.org git
tree, and compare it with the baseline on v6.16-rc7. All I have to do
to spin up some guests and also leverage a large soak duration is:

make defconfig-xfs_reflink_lbs \
	TEST_AB=y
	KDEVOPS_HOSTS_PREFIX="pw2" \
	LINUX_TREE=/mirror/mcgrof-linus.git/ \
	LINUX_TREE_REF=v6.16-rc7 \
	LINUX_DEV_TREE=/mirror/mcgrof-linus.git/
	LINUX_DEV_TREE_REF="20250723-migrate-folio-stats" \
	SOAK_DURATION=432000 -j128

After make, I see:

egrep "bootlinux_dev_tree|target_linux_git|target_linux_dev_ref|target_linux_ref" extra_vars.yaml
bootlinux_dev_tree_set_by_cli: True
bootlinux_dev_tree_ref_set_by_cli: True
bootlinux_dev_tree: "/mirror/mcgrof-linus.git/"
target_linux_dev_ref: "20250723-migrate-folio-stats"
target_linux_git: /mirror/mcgrof-linus.git/
target_linux_ref: v6.16-rc7

Now I can run *any* workflow in parallel against my two target kernels,
after make linux, fstests is just one example workflow.

Signed-off-by: Luis Chamberlain <[email protected]>

Author: mcgrof

kconfigs/Kconfig.kdevops

@@ -51,6 +51,11 @@ config KDEVOPS_HOSTS_PREFIX_SET_BY_CLI
 	bool
 	default $(shell, scripts/check-cli-set-var.sh KDEVOPS_HOSTS_PREFIX)
 
+config KDEVOPS_BASELINE_AND_DEV_SET_BY_CLI
+	bool
+	output yaml
+	default $(shell, scripts/check-cli-set-var.sh TEST_AB)
+
 config KDEVOPS_HOSTS_PREFIX
 	string "The hostname prefix to use for nodes"
 	output yaml
@@ -78,7 +83,8 @@ config KDEVOPS_CUSTOM_SSH_KEXALGORITHMS
 
 config KDEVOPS_BASELINE_AND_DEV
 	bool "Enable both a baseline and development system per target test"
-	default n
+	default n if !KDEVOPS_BASELINE_AND_DEV_SET_BY_CLI
+	default y if KDEVOPS_BASELINE_AND_DEV_SET_BY_CLI
 	help
 	  By default kdevops will only spawn a baseline target node (local
 	  virtualization or cloud node) for your Linux kernel testing and

workflows/linux/Kconfig

@@ -10,6 +10,16 @@ config BOOTLINUX_TREE_REF_SET_BY_CLI
 	output yaml
 	default $(shell, scripts/check-cli-set-var.sh LINUX_TREE_REF)
 
+config BOOTLINUX_DEV_TREE_SET_BY_CLI
+	bool
+	output yaml
+	default $(shell, scripts/check-cli-set-var.sh LINUX_DEV_TREE)
+
+config BOOTLINUX_DEV_TREE_REF_SET_BY_CLI
+	bool
+	output yaml
+	default $(shell, scripts/check-cli-set-var.sh LINUX_DEV_TREE_REF)
+
 config BOOTLINUX_HAS_PURE_IOMAP_CONFIG
 	bool
 
@@ -415,7 +425,8 @@ if BOOTLINUX_AB_DIFFERENT_REF
 config BOOTLINUX_DEV_TREE
 	string "Development kernel tree URL"
 	output yaml
-	default BOOTLINUX_TREE
+	default BOOTLINUX_TREE if !BOOTLINUX_DEV_TREE_SET_BY_CLI
+	default $(shell, ./scripts/append-makefile-vars.sh $(LINUX_DEV_TREE)) if BOOTLINUX_DEV_TREE_SET_BY_CLI
 	help
 	  Git tree URL for the development kernel. If left empty or same
 	  as the baseline tree, the same tree will be used with a different
@@ -424,7 +435,8 @@ config BOOTLINUX_DEV_TREE
 config TARGET_LINUX_DEV_REF
 	string "Development kernel reference"
 	output yaml
-	default $(shell, scripts/infer_last_stable_kernel.sh)
+	default $(shell, scripts/infer_last_stable_kernel.sh) if !BOOTLINUX_DEV_TREE_REF_SET_BY_CLI
+	default $(shell, ./scripts/append-makefile-vars.sh $(LINUX_DEV_TREE_REF)) if BOOTLINUX_DEV_TREE_REF_SET_BY_CLI
 	help
 	  Git reference (branch, tag, or commit) for the development kernel.
 	  This should be different from the baseline reference to enable