From f31341c9e26e39e5646f8c46799496d0a83da6b3 Mon Sep 17 00:00:00 2001 From: Alex Bradbury Date: Tue, 5 Nov 2024 16:59:06 +0000 Subject: [PATCH 1/7] [docs] Expand HowToAddABuilder with guidance on testing locally Once and land, it's quite reasonable to ask people to test their builder configurations locally. This patch adds documentation on how to do so. I think review at this stage is useful, but of course if there's more review feedback on it's possible some details may change. This won't be committed until those llvm-zorg PRs land of course. --- llvm/docs/HowToAddABuilder.rst | 68 ++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) diff --git a/llvm/docs/HowToAddABuilder.rst b/llvm/docs/HowToAddABuilder.rst index f07640f6627cf..060c3104b062f 100644 --- a/llvm/docs/HowToAddABuilder.rst +++ b/llvm/docs/HowToAddABuilder.rst @@ -179,6 +179,74 @@ Here are the steps you can follow to do so: buildbot.tac file to change the port number from 9994 to 9990 and start it again. +Testing a Builder Config Locally +================================ + +It's possible to test a builder running against a local version of LLVM's +buildmaster configuration. This can be helpful to allow quickly identifying +and iterating over fixes to any issues in either the changes that introduce +the new builder, or the machine configuration for your worker (preinstalled +packages etc). A buildmaster launched in this "local testing" mode will bind +only to local interfaces, use SQLite as the database, use a fixed password for +workers, and disable things like GitHub authentication. + +* Within a checkout of `llvm-zorg `_, + create and activate a Python `venv + `_ and install the necessary + dependencies. + + .. code-block:: bash + + python -m venv bbenv + source bbenv/bin/activate + pip install buildbot{,-console-view,-grid-view,-waterfall-view,-worker,-www}==3.11.7 urllib3 + +* Initialise the necessary buildmaster files, link to the configuration in + ``llvm-zorg`` and run a litmus check (run in the directory of your choice): + + .. code-block:: bash + + buildbot create-master llvm-testbbmaster + cd llvm-testbbmaster + ln -s /path/to/checkout/of/llvm-zorg/buildbot/osuosl/master/master.cfg . + ln -s /path/to/checkout/of/llvm-zorg/buildbot/osuosl/master/config/ . + ln -s /path/to/checkout/of/llvm-zorg/zorg/ . + BUILDMASTER_TEST=1 buildbot checkconfig + +* Start the buildmaster using the command below. After a few seconds to + startup, you should be able to open the web UI at ``http://localhost:8011``. + If there are any errors or this isn't working, be sure to check + ``twistd.log`` for more information. + + .. code-block:: bash + + BUILDMASTER_TEST=1 buildbot start --nodaemon . + +* With the above in place, you can now create and start a buildbot worker. + Ensure you pick the correct name for the worker attached to the build + configuration you want to test in + ``buildbot/osuosl/master/config/builders.py``. After doing the below, either + wait until the poller sets off a build, or you can force a build to start in + the web UI (which is also the best place to review the build results). + + .. code-block:: bash + + buildbot-worker create-worker \ + localhost:9990 \ + \ + test + buildbot-worker start --nodaemon + +This local testing configuration defaults to binding only to the loopback +interface for security reasons. If you want to run the test worker on a +different machine, or to run the buildmaster on a remote server, ssh port +forwarding can be used to make connection possible. For instance, if running +the buildmaster on a remote server the following command will suffice to make +the web UI accessible via ``http://localhost:8011`` and make it possible for a +local worker to connect to the remote buildmaster by connecting to +``localhost:9900``: ``ssh -N -L 8011:localhost:8011 -L 9990:localhost:9990 +username@server_address``. + Best Practices for Configuring a Fast Builder ============================================= From 15a98e1b3abafa52a4da75f3ba59c585e48cb5a6 Mon Sep 17 00:00:00 2001 From: Alex Bradbury Date: Sun, 10 Nov 2024 08:43:05 +0000 Subject: [PATCH 2/7] Address review comments --- llvm/docs/HowToAddABuilder.rst | 64 ++++++++++++++++++++-------------- 1 file changed, 37 insertions(+), 27 deletions(-) diff --git a/llvm/docs/HowToAddABuilder.rst b/llvm/docs/HowToAddABuilder.rst index 060c3104b062f..d6331238f8c32 100644 --- a/llvm/docs/HowToAddABuilder.rst +++ b/llvm/docs/HowToAddABuilder.rst @@ -182,13 +182,16 @@ Here are the steps you can follow to do so: Testing a Builder Config Locally ================================ -It's possible to test a builder running against a local version of LLVM's -buildmaster configuration. This can be helpful to allow quickly identifying -and iterating over fixes to any issues in either the changes that introduce -the new builder, or the machine configuration for your worker (preinstalled -packages etc). A buildmaster launched in this "local testing" mode will bind -only to local interfaces, use SQLite as the database, use a fixed password for -workers, and disable things like GitHub authentication. +It is possible to test a builder running against a local version of LLVM's +buildmaster configuration. This allows you to test changes to builder, +worker, and buildmaster configuration. A buildmaster launched in this "local +testing" mode will: +* Bind only to local interfaces. +* Use SQLite as the database. +* Use a single fixed password for workers. +* Disable extras like GitHub authentication. + +In order to use this "local testing" mode: * Within a checkout of `llvm-zorg `_, create and activate a Python `venv @@ -202,7 +205,8 @@ workers, and disable things like GitHub authentication. pip install buildbot{,-console-view,-grid-view,-waterfall-view,-worker,-www}==3.11.7 urllib3 * Initialise the necessary buildmaster files, link to the configuration in - ``llvm-zorg`` and run a litmus check (run in the directory of your choice): + ``llvm-zorg`` and ask ``buildbot`` to check the configuration. This step can + be run from any directory. .. code-block:: bash @@ -213,22 +217,19 @@ workers, and disable things like GitHub authentication. ln -s /path/to/checkout/of/llvm-zorg/zorg/ . BUILDMASTER_TEST=1 buildbot checkconfig -* Start the buildmaster using the command below. After a few seconds to - startup, you should be able to open the web UI at ``http://localhost:8011``. - If there are any errors or this isn't working, be sure to check - ``twistd.log`` for more information. - +* Start the buildmaster. .. code-block:: bash BUILDMASTER_TEST=1 buildbot start --nodaemon . -* With the above in place, you can now create and start a buildbot worker. - Ensure you pick the correct name for the worker attached to the build - configuration you want to test in - ``buildbot/osuosl/master/config/builders.py``. After doing the below, either - wait until the poller sets off a build, or you can force a build to start in - the web UI (which is also the best place to review the build results). +* After waiting a few seconds for startup to complete, you should be able to + open the web UI at ``http://localhost:8011``. If there are any errors or + this isn't working, check ``twistd.log`` (within the current directory) for + more information. +* You can now create and start a buildbot worker. Ensure you pick the correct + name for the worker associated with the build configuration you want to test + in ``buildbot/osuosl/master/config/builders.py``. .. code-block:: bash buildbot-worker create-worker \ @@ -237,15 +238,24 @@ workers, and disable things like GitHub authentication. test buildbot-worker start --nodaemon +* Either wait until the poller sets off a build, or alternatively force a + build to start in the web UI (which is also the best place to review the + build results). + This local testing configuration defaults to binding only to the loopback -interface for security reasons. If you want to run the test worker on a -different machine, or to run the buildmaster on a remote server, ssh port -forwarding can be used to make connection possible. For instance, if running -the buildmaster on a remote server the following command will suffice to make -the web UI accessible via ``http://localhost:8011`` and make it possible for a -local worker to connect to the remote buildmaster by connecting to -``localhost:9900``: ``ssh -N -L 8011:localhost:8011 -L 9990:localhost:9990 -username@server_address``. +interface for security reasons. + +If you want to run the test worker on a different machine, or to run the +buildmaster on a remote server, ssh port forwarding can be used to make +connection possible. For instance, if running the buildmaster on a remote +server the following command will suffice to make the web UI accessible via +``http://localhost:8011`` and make it possible for a local worker to connect +to the remote buildmaster by connecting to ``localhost:9900``: + + .. code-block:: bash + + ssh -N -L 8011:localhost:8011 -L 9990:localhost:9990 username@server_address + Best Practices for Configuring a Fast Builder ============================================= From 4950b7c2f8528a8cad680680cb22a0607d8a5ff5 Mon Sep 17 00:00:00 2001 From: Alex Bradbury Date: Sun, 10 Nov 2024 08:44:01 +0000 Subject: [PATCH 3/7] Missing blank lines --- llvm/docs/HowToAddABuilder.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/llvm/docs/HowToAddABuilder.rst b/llvm/docs/HowToAddABuilder.rst index d6331238f8c32..073c0e4ab98c1 100644 --- a/llvm/docs/HowToAddABuilder.rst +++ b/llvm/docs/HowToAddABuilder.rst @@ -218,6 +218,7 @@ In order to use this "local testing" mode: BUILDMASTER_TEST=1 buildbot checkconfig * Start the buildmaster. + .. code-block:: bash BUILDMASTER_TEST=1 buildbot start --nodaemon . @@ -230,6 +231,7 @@ In order to use this "local testing" mode: * You can now create and start a buildbot worker. Ensure you pick the correct name for the worker associated with the build configuration you want to test in ``buildbot/osuosl/master/config/builders.py``. + .. code-block:: bash buildbot-worker create-worker \ From 9afcc2970d6dd6d8aaa6c586d1719975ba6cb373 Mon Sep 17 00:00:00 2001 From: Alex Bradbury Date: Sun, 10 Nov 2024 15:49:14 +0000 Subject: [PATCH 4/7] Get rid of repeated use of 'configuration' in opening para --- llvm/docs/HowToAddABuilder.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/llvm/docs/HowToAddABuilder.rst b/llvm/docs/HowToAddABuilder.rst index 073c0e4ab98c1..8c80beaf0434e 100644 --- a/llvm/docs/HowToAddABuilder.rst +++ b/llvm/docs/HowToAddABuilder.rst @@ -183,9 +183,9 @@ Testing a Builder Config Locally ================================ It is possible to test a builder running against a local version of LLVM's -buildmaster configuration. This allows you to test changes to builder, -worker, and buildmaster configuration. A buildmaster launched in this "local -testing" mode will: +buildmaster setup. This allows you to test changes to builder, worker, and +buildmaster configuration. A buildmaster launched in this "local testing" mode +will: * Bind only to local interfaces. * Use SQLite as the database. * Use a single fixed password for workers. From 2d02173d8e747dcad8592d777f2cc9c3925789fc Mon Sep 17 00:00:00 2001 From: Alex Bradbury Date: Tue, 12 Nov 2024 12:47:33 +0000 Subject: [PATCH 5/7] Reflect rename of BUILDMASTER_TEST to BUILDBOT_TEST --- llvm/docs/HowToAddABuilder.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/llvm/docs/HowToAddABuilder.rst b/llvm/docs/HowToAddABuilder.rst index 8c80beaf0434e..0a874b69fec79 100644 --- a/llvm/docs/HowToAddABuilder.rst +++ b/llvm/docs/HowToAddABuilder.rst @@ -215,13 +215,13 @@ In order to use this "local testing" mode: ln -s /path/to/checkout/of/llvm-zorg/buildbot/osuosl/master/master.cfg . ln -s /path/to/checkout/of/llvm-zorg/buildbot/osuosl/master/config/ . ln -s /path/to/checkout/of/llvm-zorg/zorg/ . - BUILDMASTER_TEST=1 buildbot checkconfig + BUILDBOT_TEST=1 buildbot checkconfig * Start the buildmaster. .. code-block:: bash - BUILDMASTER_TEST=1 buildbot start --nodaemon . + BUILDBOT_TEST=1 buildbot start --nodaemon . * After waiting a few seconds for startup to complete, you should be able to open the web UI at ``http://localhost:8011``. If there are any errors or From bb58bf8b4e603a1329417f7cc9ab841b7c22460e Mon Sep 17 00:00:00 2001 From: Alex Bradbury Date: Tue, 12 Nov 2024 12:48:57 +0000 Subject: [PATCH 6/7] Split out extra bullet point as per suggestion --- llvm/docs/HowToAddABuilder.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/llvm/docs/HowToAddABuilder.rst b/llvm/docs/HowToAddABuilder.rst index 0a874b69fec79..0df5d6b1cb7a1 100644 --- a/llvm/docs/HowToAddABuilder.rst +++ b/llvm/docs/HowToAddABuilder.rst @@ -241,8 +241,9 @@ In order to use this "local testing" mode: buildbot-worker start --nodaemon * Either wait until the poller sets off a build, or alternatively force a - build to start in the web UI (which is also the best place to review the - build results). + build to start in the web UI. + +* Review the progress and results of the build in the web UI. This local testing configuration defaults to binding only to the loopback interface for security reasons. From 4c0a0796c20949d556c944e3814b9bdf799e6247 Mon Sep 17 00:00:00 2001 From: Alex Bradbury Date: Tue, 12 Nov 2024 12:49:22 +0000 Subject: [PATCH 7/7] Clarify server_address --- llvm/docs/HowToAddABuilder.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/llvm/docs/HowToAddABuilder.rst b/llvm/docs/HowToAddABuilder.rst index 0df5d6b1cb7a1..5cf1291bcd11b 100644 --- a/llvm/docs/HowToAddABuilder.rst +++ b/llvm/docs/HowToAddABuilder.rst @@ -257,7 +257,7 @@ to the remote buildmaster by connecting to ``localhost:9900``: .. code-block:: bash - ssh -N -L 8011:localhost:8011 -L 9990:localhost:9990 username@server_address + ssh -N -L 8011:localhost:8011 -L 9990:localhost:9990 username@buildmaster_server_address Best Practices for Configuring a Fast Builder