Development Build

Commits

• feed2ae: Fix psutil cpu_percent return 0.0 issue. (Yuxuan Cai) #1278

25.8.0

.. _rev_25.8.0:

25.8.0 (2025-08-15)

• :changed:Changed Fix an issue where PDFs failed to render in certain cases. Fix console output of certain assertion details.

• :changed:Changed Upgrade gherkin-official to 32 for supporting BDD on Python 3.11 or later.

  Note that spaces are no longer allowed in gherkin tags, this could be a breaking change for BDD users. As a result, lines like @tag a @tag b in feature files should be replaced with line like @tag_a @tag_b, and line @KNOWN_TO_FAIL: some reason could be replaced with @KNOWN_TO_FAIL:some_reason.

• 🆕New Added a timeout entry to the report to make testplan-level timeout events more visible to users. Fixed an issue where multitests incorrectly appeared as passed in stdout upon timeout. Updated the status of unfinished test suites and multitests to "incomplete".

• :changed:Changed Speed up interactive mode report initial load.

• 🆕New Create new frontend components DictMatchAll and FixMatchAll.

• :changed:Changed The wrapped driver class within :py:class:RemoteDriver <testplan.common.remote.remote_driver.RemoteDriver> is now constructed only on remote host.

• :changed:Changed Method signature check of testcases/test hooks has been relaxed to allow underscore-prefixed variants of parameters, so that linter wouldn't raise unused variable warnings on them.

• :changed:Changed Move certain dependencies to extras (i.e. optional dependencies) to shrink the installation size in pipelines. User need to install Testplan with extras in order to use certain features.

  • Extra interactive for interactive mode & report display through local server features, containing flask & several related packages.
  • Extra plotly for plotting with plotly feature, containing pandas & plotly.
  • Extra all for all the features above.

  For example, to use interactive mode feature, one can install Testplan with the following command:

  .. code-block:: bash

  pip install testplan[interactive]
  

• :removed:Removed Remove support for Python 3.7 and 3.8.

• :changed:Changed Parameters auto_part_runtime_limit and plan_runtime_target of @test_plan now takes "auto". See :ref:documentation <auto_part> for details.

• :changed:Changed Make teardown_time field mandatory in the json file provided via --runtime-data cmdline arg for auto part and smart scheduling.

• :changed:Changed Fix an issue where nested lists in dict.match and dict.match_all assertions were not fully shown in the result output.

• :changed:Changed Temporarily restrict package dependency marshmallow to versions <4.

• :changed:Changed Fixed fake report for UI development

• :changed:Changed Runtime data JSON file specified through --runtime-data now accepts an optional testcase_count field, which should hold the total number of testcases from the previous run. If testcase_count is specified, estimated MultiTest execution time will be adjusted by the ratio of current run's testcase count to that of previous run, which should bring more accurate auto-partitioning and improved testing resource utilization.

• :changed:Changed Set timezone entries in Testplan reports to UTC when local timezone detection fails.

• :changed:Changed Fix an issue that generated incorrect figures of driver starting/stopping timeline inside the driver-info feature.

25.3.0

.. _rev_25.3.0:

25.3.0 (2025-03-25)

• :changed:Changed Change Testplan exported JSON report structure to reduce report size.

  • Remove unused report entry fields.
    • fix_spec_path.
    • status_override and status_reason in case they are empty.
    • line_no, code_context and file_path if --code is not enabled.
    • env_status, part, strict_order and host depending on report category.
  • Remove unused assertion entry fields category and flag if they are DEFAULT.
  • Merge assertion entry fields utc_time and machine_time into a unix timestamp field timestamp, and store timezone info in parent Test-level report under key timezone.
  • Replace ISO 8601 time string with unix timestamp in all timer fields, and add a timezone field to Testplan-level report as well.
  • Update data structure of several serialized assertion entries.
    • Delta encode level info of flattened_dict fields of DictLog and FixLog entries.
    • Delta encode level info of comparison fields of DictMatch and FixMatch entries.
    • Delta encode level info of nested comparison fields of DictMatchAll and FixMatchAll entries, remove extra nesting of matches as well.
    • Preserve abbreviations of match status of DictMatch, FixMatch, DictMatchAll and FixMatchAll entries, i.e. p instead of Passed, f instead of Failed, i instead of Ignored.
    • Remove indices field of TableLog entries.

• :changed:Changed Fix releaseherald documentation

• :deprecated:Deprecated Support for Python 3.7 and 3.8 is deprecated and will be removed soon.

• :changed:Changed Handle potential race condition during resource monitor termination

• :changed:Changed Improve error logging for :py:class:~testplan.common.remote.remote_service.RemoteService; fix incorrect imitated workspace on remote due to leftover symlink from previous run.

• :changed:Changed Remove Sphinx and other packages for building document from Testplan's dependencies.

• :changed:Changed Copy permission bits for the copied binary in App.

25.1.0

.. _rev_25.1.0:

25.1.0 (2025-01-20)

• :changed:Changed Checks if a process exists by reading the /proc/<pid>/stat.

• :changed:Changed Support :py:class:RemoteDriver <testplan.common.remote.remote_driver.RemoteDriver> in dependency graph of test environment (the dependencies parameter).

• :changed:Changed Use lazy import for Matplotlib and move cache to runpath.

• 🆕New Added --code flag to collect code context for the assertions. Code context one-liner will be displayed on the web UI if enabled.
  Note that file path information is no longer collected by default. To collect file path information, enable code context.

• 🆕New Add a new summary page on resource view to show the task allocation per host.

• :changed:Changed Refactor the stop logic of :py:class:App <~testplan.testing.multitest.driver.app.App> driver for faster environment shutdown. Rename parameter sigint_timeout to stop_timeout. Add a new parameter stop_signal for custom stop signals, its default value None invokes terminate method to stop subprocess, i.e. sending SIGTERM signal to subprocess on Linux.
  Environment will fail to stop if subprocess doesn't terminate within the default 5-second stop_timeout for graceful shutdown. Increase stop_timeout or change stop_signal (to maybe SIGKILL on Linux) could resolve this issue.

• :changed:Changed Make sure when stop() is called on App type driver, we clean up all orphaned processes.

• :changed:Changed If :py:class:App <~testplan.testing.multitest.driver.app.App> driver times out during shutdown or leaves orphaned processes after shutdown, Testplan will now emit a warning and perform a forced cleanup instead of failing the tests.

• :changed:Changed Increase the number of Remote worker setup thread.

• :changed:Changed Fix interactive mode crashing issue when loading a namespace package. (It is still not supported to reload namespace packages.) SyntaxError will no longer be suppressed during interactive mode code reloading.

• :changed:Changed Fix incorrect early stop detection logic.

• :changed:Changed Swapped Run and Reload buttons on the interactive UI by user request.

• :changed:Changed Use a new JSON library orjson to improve performance when using Python 3.8 or later versions.

• :changed:Changed Limit the length of parameterization testcase name to 255 characters. If the name length exceeds 255 characters, index-suffixed names (e.g., {func_name} 1, {func_name} 2) will be used.

• 🆕New Testplan now includes its own version in generated report.

• :changed:Changed JSONExporter will log a "file not found" warning in the log instead of raising an exception.

• :changed:Changed Fixed an issue where enabling Status icons crashed the report when a test was marked as XFAIL.

• :changed:Changed Update orjson dumping option to allow serializing numpy objects.

24.9.2

.. _rev_24.9.2:

24.9.2 (2024-09-12)