docs: more docs for testing and repackaging existing software (#1645)

Author: wolfv

docs/reference/recipe_file.md

@@ -698,7 +698,6 @@ Using a runtime dependency name:
 !!! note
     `ignore_run_exports` only applies to runtime dependencies coming from an upstream package.
 
-
 ## Tests section
 
 `rattler-build` supports four different types of tests. The "script test" installs
@@ -744,6 +743,19 @@ tests:
       - bspatch4 -h
 ```
 
+#### External scripts
+
+You can also easily run a script from your recipe directory.
+Note that your package should either depend on the interpreter (e.g. Python or R)
+or you need to add a `requirements` section to the test that installs the interpreter.
+
+```yaml
+tests:
+  - script: tests/run_test.py
+  - script: tests/run_test.R
+  - script: tests/run_test.sh
+```
+
 #### Extra test files
 
 Test files that are copied from the source work directory into the temporary
@@ -815,6 +827,41 @@ import bsdiff4
 import bspatch4
 ```
 
+### Perl tests
+
+For this test type you can list a set of Perl modules that need to be
+importable. The test will fail if any of the modules cannot be imported.
+
+```yaml
+tests:
+  - perl:
+      uses:
+        - Call::Context
+```
+
+Internally this will write a small Perl script that imports the modules:
+
+```perl
+use Call::Context;
+```
+
+### R tests
+
+For this test type you can list a set of R modules that need to be
+importable. The test will fail if any of the modules cannot be imported.
+
+```yaml
+- r:
+    libraries:
+      - knitr
+```
+
+Internally this will write a small R script that imports the modules:
+
+```r
+library(knitr)
+```
+
 ### Check for package contents
 
 Checks if the built package contains the mentioned items. These checks are executed directly at

docs/testing.md

@@ -1,83 +1,122 @@
 # Testing packages
 
 When you are developing a package, you should write tests for it. The tests are
-automatically executed right after the package build has finished.
+automatically executed as soon as the package build and all it's run dependencies
+are ready.
 
-The tests from the test section are actually packaged _into_ your package and
-can also be executed straight from the existing package.
+## Writing tests
 
-The idea behind adding the tests into the package is that you can execute the
-tests independently from building the package. That is also why we are shipping
-a `test` subcommand that takes as input an existing package and executes the
-tests:
+You can add one or more tests to your package in the `tests` section of the recipe (or output).
+Each test is run independently, in a separate environment.
 
-```bash
-rattler-build test --package-file ./xtensor-0.24.6-h60d57d3_0.tar.bz2
-```
-
-Running the above command will extract the package and create a clean
-environment where the package and dependencies are installed. Then the tests are
-executed in this newly-created environment.
+One notable difference are the `package_contents` tests that are executed right after the package
+is prepared and do not create a new environment (as we only analyze the contents of the package).
 
-If you inspect the package contents, you would find the test files under
-`info/test/*`.
+```yaml title="recipe.yaml"
+tests:
+  # commands to run to test the package. If any of the commands
+  # returns with an error code, the test is considered failed.
+  - script:
+      - echo "Hello world"
+      - exit 1  # this will fail the test
 
-## How tests are translated
+  # run a script from the recipe folder
+  - script: sometest.py
 
-The `tests` section allows you to specify the following things:
+  # run a Python script with the Python interpreter
+  - script:
+      interpreter: python
+      content: |
+        import mypkg
+        assert mypkg.__version__ == "0.1.0"
 
-```yaml
-tests:
+  # execute `pytest` with the tests from the `tests` folder
   - script:
-      # commands to run to test the package. If any of the commands
-      # returns with an error code, the test is considered failed.
-      - echo "Hello world"
       - pytest ./tests
-
     # additional requirements at test time
     requirements:
       run:
         - pytest
-
+        - python 3.9.*  # require an older python version
+    # extra files to be copied to the test folder from the recipe or source directory
     files:
-      # Extra files to be copied to the test directory from the "work directory"
-      source:
-        - tests/
-        - test.py
-        - *.sh
       recipe:
-        - more_tests/*.py
+        - tests/
 
-  # This test section tries to import the Python modules and errors if it can't
+  # python specific tests
   - python:
+      # this test section tries to import the python modules and errors if it can't
       imports:
         - mypkg
-        - mypkg.subpkg
+      pip_check: true
+      python_version: [3.9.*, 3.10.*]  # run against multiple older python versions
+
+  - r:
+      libraries:
+        - dplyr
+
+  - perl:
+      modules:
+        - JSON
+
+  # test the contents of the package. If any of the globs does not match _any_ file, the test fails.
+  - package_contents:
+      files:
+        - share/package/*.txt
 ```
 
-When you are writing a test for your package, additional files are created and
-added to your package. These files are placed under the `info/tests/{index}/`
-folder for each test.
+### Testing package contents
+
+The `package_contents` test is a special test that is executed right after the
+package is prepared. It does not create a new environment, but instead checks the paths that will be part of the final package.
+It can be very useful as a "sanity check" to ensure that the package contains the expected files.
+
+It has multiple sub-keys that help when building cross-platform packages:
+
+- **`files`**: a list of globs that should match at least one file in the package. If any of the globs do not match, the test fails.
+- **`lib`**: matches libraries in the package (`.so`, `.dll`, `.dylib` files). The test fails if any of the libraries are not found. It's enough to specify the library name without any extension (e.g. `foo` will match `libfoo.so`, `libfoo.dylib`, and `foo.dll`).
+- **`include`**: matches files under the `include` directory in the package. You can specify the file name like `foo.h`.
+- **`bin`**: matches files under the `bin` directory in the package. You can specify executable names like `foo` which will match `foo.exe` on Windows and `foo` on Linux and macOS.
+- **`site_packages`**: matches files under the `site-packages` directory in the package. You can specify the import path like `foobar.api` which will match `foobar/api.py` and `foobar/api/__init__.py`.
+
+## Testing existing packages
 
-For a script test:
+The tests from the test section are actually added _into_ your package and
+can also be executed straight from the existing package.
+
+The idea behind adding the tests into the package is that you can execute the
+tests independently from building the package. That is also why we are shipping
+a `test` subcommand that takes as input an existing package and executes the
+tests:
 
-- All the files are copied straight into the test folder (under
-  `info/tests/{index}/`)
-- The script is turned into a `run_test.sh` or `run_test.bat` file
-- The extra requirements are stored as a JSON file called
-  `test_time_dependencies.json`
+```bash
+rattler-build test --package-file ./xtensor-0.24.6-h60d57d3_0.tar.bz2
+```
+
+Running the above command will extract the package and create a clean
+environment where the package and dependencies are installed. Then the tests are
+executed in this newly-created environment.
+
+If you inspect the package contents, you would find the test files under
+`info/test/*`.
+
+## How tests are translated
 
-For a Python import test:
+The `tests` section allows you to define test configurations for your package.
+Tests are serialized to `info/tests/tests.yaml` in the created package and read from there during test execution.
 
-- A JSON file is created that is called `python_test.json` and stores the
-  imports to be tested and whether to execute `pip check` or not. This file is
-  placed under `info/tests/{index}/`
+When adding extra files to your tests:
 
-For a downstream test:
+1. **During package creation**
+     - Files are copied to `$PREFIX/etc/conda/test-files/{pkg_name}/{idx}`
+     - `{idx}` is a sequential number assigned to each test
+     - Files can come from both `source` (work directory) and `recipe` locations
+2. **During test execution**
+     - Files are copied from `$PREFIX/etc/conda/test-files/{pkg_name}/{idx}` to a temporary directory
+     - Tests run within this temporary directory
+     - Use relative paths to access these files in your test commands
 
-- A JSON file is created that is called `downstream_test.json` and stores the
-  downstream tests to be executed. This file is placed under
-  `info/tests/{index}/`
+This approach ensures test files are properly packaged and available during test execution.
 
 ## Legacy tests
 

docs/tutorials/python.md

@@ -87,7 +87,9 @@ rattler-build build --recipe ./ipywidgets
 
 We will build a package for `numpy` – which contains compiled code.
 Since compiled code is `python` version-specific, we will need to specify the `python` version explicitly.
-The best way to do this is with a "variant_config.yaml" file:
+
+The best way to do this is with a "variant_config.yaml" file.
+The variant config file allows us to easily compile the package against multiple Python versions.
 
 ```yaml title="variants.yaml"
 python:

docs/tutorials/repackaging.md

@@ -0,0 +1,79 @@
+# Repackaging existing software
+
+It's totally possible to repackage existing software using rattler-build, and make it easy to install
+with conda, mamba or pixi.
+
+Repackaging existing binaries is not recommended on `conda-forge`, but totally acceptable for your own channels / repositories.
+
+## Example for `linkerd`
+
+This example shows how to repackage the `linkerd` binary. The `linkerd` binary is a command line tool that is used to manage and monitor Kubernetes clusters, and a pre-built binary is available for download from Github releases. Alternatively, you could also follow the Go packaging tutorial to build linkerd from source!
+
+```yaml
+package:
+  name: linkerd
+  version: 25.5.2
+
+source:
+  - if: target_platform == "linux-64"
+    then:
+      url: https://github.com/linkerd/linkerd2/releases/download/edge-25.5.2/linkerd2-cli-edge-25.5.2-linux-amd64
+      sha256: 55e7721ab0eb48217f239628b55517b7d663a962df18cdab180e5d42e45f83cb
+      file_name: linkerd
+  - if: target_platform == "osx-arm64"
+    then:
+      url: https://github.com/linkerd/linkerd2/releases/download/edge-25.5.2/linkerd2-cli-edge-25.5.2-darwin-arm64
+      sha256: 405ddf3af0089bfece93d811c9bfb9f63e3a000e3f423163fc56690ef4d427cf
+      file_name: linkerd
+  # To support other platforms you can add more `if` statements here
+
+build:
+  script:
+    # make linkerd binary executable
+    - chmod +x linkerd
+    # make sure that the `$PREFIX/bin` directory exists
+    - mkdir -p $PREFIX/bin
+    # move or copy the binary to the `$PREFIX/bin` directory
+    - mv linkerd $PREFIX/bin/
+
+tests:
+  - script:
+      - linkerd version
+      # you can add more tests here
+
+about:
+  homepage: https://linkerd.io/
+  license: Apache-2.0
+  summary: Linkerd is an ultralight service mesh for Kubernetes.
+  description: |
+    Linkerd is an ultralight service mesh for Kubernetes.
+    It adds observability, reliability, and security to your
+    applications without requiring any code changes.
+    Linkerd is open source and free to use.
+  # Note: since we are downloading a binary, we don't have a license file.
+  # You can put the license in the recipe directory, and it will be picked up from there.
+  license_file: LICENSE
+  # documentation: ...
+  repository: https://github.com/linkerd/linkerd2
+```
+
+!!! note
+
+    To repackage the `linkerd` package on `osx-arm64` for `linux-64`, you can pass the `--target-platform` argument to `rattler-build`:
+
+    ```bash
+    rattler-build build --target-platform linux-64 linkerd
+    ```
+
+## Adding system requirements
+
+Some packages have system requirements (e.g. on `glibc` on Linux, or the macOS SDK on macOS).
+
+You can add system requirements like this to the `run` section by depending on virtual packages:
+
+```yaml
+requirements:
+  run:
+    - ${{ "__glibc >=2.17" if linux }}
+    - ${{ "__osx >=10.15" if osx }}
+```

mkdocs.yml

@@ -105,6 +105,7 @@ nav:
     - "Go": tutorials/go.md
     - "Perl": tutorials/perl.md
     - "R": tutorials/r.md
+    - "Repackaging": tutorials/repackaging.md
     - "Converting from conda-build": converting_from_conda_build.md
 
   - Build options: