Add post-release validation workflow (#2058)

* Add post-release validation workflow

* reduce triggers for pr workflows

* temp change for testing

* Fix installation mistakes

* Use venv

* Revert "temp change for testing"

This reverts commit 4b7edb9.

* Revert "reduce triggers for pr workflows"

This reverts commit 992125d.

* Fix typo

* Add links to description in Release Process

* Update .github/actions/run_examples_using_pypi_uploads/action.yml

Capitalize PyPI

Co-authored-by: Marcos <[email protected]>

* Apply suggestions from code review

Give examples of action parameter values

Co-authored-by: Marcos <[email protected]>

* Apply suggestions from code review

Capitalize PyPI

---------

Co-authored-by: Marcos <[email protected]>

Author: ni-jfitzger

.github/actions/run_examples_using_pypi_uploads/action.yml

@@ -0,0 +1,38 @@
+name: Run examples using PyPI uploads
+description: Install module(s) from PyPI and run examples. Intended for release validation.
+inputs:
+  module_name:
+    description: module name (e.g. "nidcpower")
+    required: true
+    default: ""
+  module_version:
+    description: module version (e.g. "1.8.1")
+    required: true
+    default: ""
+
+runs:
+  using: composite
+  steps:
+    # The local wheel installation and single environment would have made using tox-system_tests.ini difficult.
+    # To keep things simple, we just use pip and pytest directly for a single Python version.
+    - run: py -3.12 -m venv --clear .venv
+      shell: powershell
+    - run: |
+        .venv\Scripts\Activate.ps1
+        python -m pip install ${{ inputs.module_name }}==${{ inputs.module_version }} pytest
+      shell: powershell
+    - if: ${{ inputs.module_name == 'nitclk' }}
+      run: |
+        .venv\Scripts\Activate.ps1
+        python -m pip install niscope==${{ inputs.module_version }}
+      shell: powershell
+    - if: ${{ inputs.module_name == 'niscope' }}
+      run: |
+        .venv\Scripts\Activate.ps1
+        python -m pip install numpy
+      shell: powershell
+    - run: |
+        .venv\Scripts\Activate.ps1
+        cd src/${{ inputs.module_name }}
+        python -m pytest ./examples -v
+      shell: powershell

.github/workflows/check_latest_release.yml

@@ -0,0 +1,48 @@
+name: check_latest_release
+
+on:
+  release:
+    types: [released]
+
+  # Allows you to run this workflow manually from the Actions tab
+  workflow_dispatch:
+    inputs:
+      release_tag:
+        description: The GitHub release tag corresponding to the PyPI releases to test
+        required: true
+        default: non-existent-tag
+  
+jobs:
+  install-module-and-run-examples:
+    name: example_test
+    if: github.repository == 'ni/nimi-python'
+    # Use win32 nimibot
+    # win64 already handles post-commit testing for coverage
+    # linux doesn't support all modules
+    runs-on:
+      - self-hosted
+      - windows
+      - x64
+      - rdss-nimibot-win-10-py32
+    timeout-minutes: 30
+    strategy:
+      matrix:
+        module_name:
+          - nidigital
+          - nitclk
+          - nifgen
+          - nidcpower
+          - nidmm
+          - niscope
+          - nimodinst
+          - nise
+          - niswitch
+    steps:
+      - name: checkout repository
+        uses: actions/checkout@v3
+      # NOTE: we don't upload test coverage for this
+      - name: run examples using PyPI uploads
+        uses: ./.github/actions/run_examples_using_pypi_uploads
+        with:
+          module_name: ${{ matrix.module_name }}
+          module_version: ${{ github.event_name == 'workflow_dispatch' && inputs.release_tag || github.event.release.tag_name }}

CONTRIBUTING.md

@@ -172,6 +172,7 @@ Release Process
     1. Merge the pull request to origin/master
     1. Create a release on GitHub using the portion from the changelog for this release for the description
         * Add the ZIP files under `generated/examples` for each module as a release artifact.
+        * This should trigger the [check_latest_release](.github/workflows/check_latest_release.yml) workflow. Check the [results](https://github.com/ni/nimi-python/actions/workflows/check_latest_release.yml) before continuing.
 1. Post-Release Steps
     1. Create and checkout another branch for post-release changes
     1. Update the module versions