feat(pytest-bdd): add support for 6.1+ (#6214)

Enable support for `pytest-bdd >= 6.1` by:
* Using `importlib_metadata` or `importlib.metadata` as necessary to
gather package version
* Switching to collection `step_func_args` after steps finish (or after
they error)
* Introducing a "safe" JSONification of `step_func_args` that catches
possible `TypeError` exceptions and returns the error in a
`json.dumps()`-safe dict

Compatibility with `pytest-bdd < 4` is maintained by moving the old
codepath to a utility function and using it whenever it looks like we
have the old-style functions available. Technically, this maintains
compatibility with `< 6.1` because that's when the latter function was
removed but, based on testing, `step_func_args` is populated as expected
in `>= 4.0` and our test suites pass for Python `3.6+`.

The backwards-compatible code is necessary because older versions of
`pytest-bdd` include fixtures in the `step_func_args` passed to the
`pytest_bdd_after_step` and `pytest_bdd_step_error`, breaking
`json.dumps()`.

Also in this change:
* Updating dependencies for testing environments
* Returning `pytest-bdd` versions in `riotfile` to their state pre #4492
.

Fixes #4492 .

## Checklist

- [x] Change(s) are motivated and described in the PR description.
- [x] Testing strategy is described if automated tests are not included
in the PR.
- [x] Risk is outlined (performance impact, potential for breakage,
maintainability, etc).
- [x] Change is maintainable (easy to change, telemetry, documentation).
- [x] [Library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
are followed. If no release note is required, add label
`changelog/no-changelog`.
- [x] Documentation is included (in-code, generated user docs, [public
corp docs](https://github.com/DataDog/documentation/)).
- [x] Backport labels are set (if
[applicable](../docs/contributing.rst#release-branch-maintenance))

## Reviewer Checklist

- [x] Title is accurate.
- [x] No unnecessary changes are introduced.
- [x] Description motivates each change.
- [x] Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes unless absolutely necessary.
- [x] Testing strategy adequately addresses listed risk(s).
- [x] Change is maintainable (easy to change, telemetry, documentation).
- [x] Release note makes sense to a user of the library.
- [x] Reviewer has explicitly acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment.
- [x] Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](../docs/contributing.rst#release-branch-maintenance)

---------

Co-authored-by: Emmett Butler <[email protected]>
Co-authored-by: Munir Abdinur <[email protected]>
Co-authored-by: Brett Langdon <[email protected]>

Author: 4 people

.riot/requirements/19a842e.txt renamed to .riot/requirements/1080b30.txt

@@ -2,32 +2,32 @@
 # This file is autogenerated by pip-compile with Python 3.7
 # by the following command:
 #
-#    pip-compile --no-annotate --resolver=backtracking .riot/requirements/19a842e.in
+#    pip-compile --no-annotate --resolver=backtracking .riot/requirements/1080b30.in
 #
-attrs==22.2.0
-coverage[toml]==7.2.2
-exceptiongroup==1.1.1
+attrs==23.1.0
+coverage[toml]==7.2.7
+exceptiongroup==1.1.2
 glob2==0.7
 hypothesis==6.45.0
-importlib-metadata==6.0.0
+importlib-metadata==6.7.0
 iniconfig==2.0.0
 mako==1.2.4
-markupsafe==2.1.2
-mock==5.0.1
+markupsafe==2.1.3
+mock==5.0.2
 more-itertools==8.10.0
 msgpack==1.0.5
 opentracing==2.4.0
-packaging==23.0
-parse==1.19.0
-parse-type==0.6.0
-pluggy==1.0.0
+packaging==23.1
+parse==1.19.1
+parse-type==0.6.2
+pluggy==1.2.0
 py==1.11.0
-pytest==7.2.2
+pytest==7.4.0
 pytest-bdd==4.1.0
-pytest-cov==4.0.0
-pytest-mock==3.10.0
+pytest-cov==4.1.0
+pytest-mock==3.11.1
 six==1.16.0
 sortedcontainers==2.4.0
 tomli==2.0.1
-typing-extensions==4.7.0
+typing-extensions==4.7.1
 zipp==3.15.0

.riot/requirements/fc4505d.txt renamed to .riot/requirements/11ca4d2.txt

@@ -2,27 +2,26 @@
 # This file is autogenerated by pip-compile with Python 3.11
 # by the following command:
 #
-#    pip-compile --no-annotate --resolver=backtracking .riot/requirements/fc4505d.in
+#    pip-compile --no-annotate --resolver=backtracking .riot/requirements/11ca4d2.in
 #
-attrs==22.2.0
-coverage[toml]==7.2.2
-glob2==0.7
+attrs==23.1.0
+coverage[toml]==7.2.7
 hypothesis==6.45.0
 iniconfig==2.0.0
 mako==1.2.4
-markupsafe==2.1.2
-mock==5.0.1
+markupsafe==2.1.3
+mock==5.0.2
 more-itertools==8.10.0
 msgpack==1.0.5
 opentracing==2.4.0
-packaging==23.0
-parse==1.19.0
-parse-type==0.6.0
-pluggy==1.0.0
-py==1.11.0
-pytest==7.2.2
-pytest-bdd==6.0.1
-pytest-cov==4.0.0
-pytest-mock==3.10.0
+packaging==23.1
+parse==1.19.1
+parse-type==0.6.2
+pluggy==1.2.0
+pytest==7.4.0
+pytest-bdd==6.1.1
+pytest-cov==4.1.0
+pytest-mock==3.11.1
 six==1.16.0
 sortedcontainers==2.4.0
+typing-extensions==4.7.1

.riot/requirements/146bb98.txt

@@ -0,0 +1,27 @@
+#
+# This file is autogenerated by pip-compile with Python 3.11
+# by the following command:
+#
+#    pip-compile --no-annotate --resolver=backtracking .riot/requirements/146bb98.in
+#
+attrs==23.1.0
+coverage[toml]==7.2.7
+hypothesis==6.45.0
+iniconfig==2.0.0
+mako==1.2.4
+markupsafe==2.1.3
+mock==5.0.2
+more-itertools==8.10.0
+msgpack==1.0.5
+opentracing==2.4.0
+packaging==23.1
+parse==1.19.1
+parse-type==0.6.2
+pluggy==1.2.0
+pytest==7.4.0
+pytest-bdd==6.1.1
+pytest-cov==4.1.0
+pytest-mock==3.11.1
+six==1.16.0
+sortedcontainers==2.4.0
+typing-extensions==4.7.1

.riot/requirements/3e8d84b.txt renamed to .riot/requirements/16a7863.txt

@@ -2,29 +2,28 @@
 # This file is autogenerated by pip-compile with Python 3.8
 # by the following command:
 #
-#    pip-compile --no-annotate --resolver=backtracking .riot/requirements/3e8d84b.in
+#    pip-compile --no-annotate --resolver=backtracking .riot/requirements/16a7863.in
 #
-attrs==22.2.0
-coverage[toml]==7.2.2
-exceptiongroup==1.1.1
-glob2==0.7
+attrs==23.1.0
+coverage[toml]==7.2.7
+exceptiongroup==1.1.2
 hypothesis==6.45.0
 iniconfig==2.0.0
 mako==1.2.4
-markupsafe==2.1.2
-mock==5.0.1
+markupsafe==2.1.3
+mock==5.0.2
 more-itertools==8.10.0
 msgpack==1.0.5
 opentracing==2.4.0
-packaging==23.0
-parse==1.19.0
-parse-type==0.6.0
-pluggy==1.0.0
-py==1.11.0
-pytest==7.2.2
-pytest-bdd==6.0.1
-pytest-cov==4.0.0
-pytest-mock==3.10.0
+packaging==23.1
+parse==1.19.1
+parse-type==0.6.2
+pluggy==1.2.0
+pytest==7.4.0
+pytest-bdd==6.1.1
+pytest-cov==4.1.0
+pytest-mock==3.11.1
 six==1.16.0
 sortedcontainers==2.4.0
 tomli==2.0.1
+typing-extensions==4.7.1

.riot/requirements/72a8638.txt renamed to .riot/requirements/16a9c86.txt

@@ -2,29 +2,28 @@
 # This file is autogenerated by pip-compile with Python 3.9
 # by the following command:
 #
-#    pip-compile --no-annotate --resolver=backtracking .riot/requirements/72a8638.in
+#    pip-compile --no-annotate --resolver=backtracking .riot/requirements/16a9c86.in
 #
-attrs==22.2.0
-coverage[toml]==7.2.2
-exceptiongroup==1.1.1
-glob2==0.7
+attrs==23.1.0
+coverage[toml]==7.2.7
+exceptiongroup==1.1.2
 hypothesis==6.45.0
 iniconfig==2.0.0
 mako==1.2.4
-markupsafe==2.1.2
-mock==5.0.1
+markupsafe==2.1.3
+mock==5.0.2
 more-itertools==8.10.0
 msgpack==1.0.5
 opentracing==2.4.0
-packaging==23.0
-parse==1.19.0
-parse-type==0.6.0
-pluggy==1.0.0
-py==1.11.0
-pytest==7.2.2
-pytest-bdd==6.0.1
-pytest-cov==4.0.0
-pytest-mock==3.10.0
+packaging==23.1
+parse==1.19.1
+parse-type==0.6.2
+pluggy==1.2.0
+pytest==7.4.0
+pytest-bdd==6.1.1
+pytest-cov==4.1.0
+pytest-mock==3.11.1
 six==1.16.0
 sortedcontainers==2.4.0
 tomli==2.0.1
+typing-extensions==4.7.1

.riot/requirements/4654b0c.txt renamed to .riot/requirements/16bf107.txt

@@ -2,32 +2,30 @@
 # This file is autogenerated by pip-compile with Python 3.7
 # by the following command:
 #
-#    pip-compile --no-annotate --resolver=backtracking .riot/requirements/4654b0c.in
+#    pip-compile --no-annotate --resolver=backtracking .riot/requirements/16bf107.in
 #
-attrs==22.2.0
-coverage[toml]==7.2.2
-exceptiongroup==1.1.1
-glob2==0.7
+attrs==23.1.0
+coverage[toml]==7.2.7
+exceptiongroup==1.1.2
 hypothesis==6.45.0
-importlib-metadata==6.0.0
+importlib-metadata==6.7.0
 iniconfig==2.0.0
 mako==1.2.4
-markupsafe==2.1.2
-mock==5.0.1
+markupsafe==2.1.3
+mock==5.0.2
 more-itertools==8.10.0
 msgpack==1.0.5
 opentracing==2.4.0
-packaging==23.0
-parse==1.19.0
-parse-type==0.6.0
-pluggy==1.0.0
-py==1.11.0
-pytest==7.2.2
-pytest-bdd==6.0.1
-pytest-cov==4.0.0
-pytest-mock==3.10.0
+packaging==23.1
+parse==1.19.1
+parse-type==0.6.2
+pluggy==1.2.0
+pytest==7.4.0
+pytest-bdd==6.1.1
+pytest-cov==4.1.0
+pytest-mock==3.11.1
 six==1.16.0
 sortedcontainers==2.4.0
 tomli==2.0.1
-typing-extensions==4.7.0
+typing-extensions==4.7.1
 zipp==3.15.0

.riot/requirements/1646d28.txt renamed to .riot/requirements/1895369.txt

@@ -2,29 +2,30 @@
 # This file is autogenerated by pip-compile with Python 3.9
 # by the following command:
 #
-#    pip-compile --no-annotate --resolver=backtracking .riot/requirements/1646d28.in
+#    pip-compile --no-annotate --resolver=backtracking .riot/requirements/1895369.in
 #
-attrs==22.2.0
-coverage[toml]==7.2.2
-exceptiongroup==1.1.1
+attrs==23.1.0
+coverage[toml]==7.2.7
+exceptiongroup==1.1.2
 glob2==0.7
 hypothesis==6.45.0
 iniconfig==2.0.0
 mako==1.2.4
-markupsafe==2.1.2
-mock==5.0.1
+markupsafe==2.1.3
+mock==5.0.2
 more-itertools==8.10.0
 msgpack==1.0.5
 opentracing==2.4.0
-packaging==23.0
-parse==1.19.0
-parse-type==0.6.0
-pluggy==1.0.0
+packaging==23.1
+parse==1.19.1
+parse-type==0.6.2
+pluggy==1.2.0
 py==1.11.0
-pytest==7.2.2
+pytest==7.4.0
 pytest-bdd==4.1.0
-pytest-cov==4.0.0
-pytest-mock==3.10.0
+pytest-cov==4.1.0
+pytest-mock==3.11.1
 six==1.16.0
 sortedcontainers==2.4.0
 tomli==2.0.1
+typing-extensions==4.7.1

.riot/requirements/19a9846.txt

@@ -6,7 +6,7 @@
 #
 atomicwrites==1.4.1
 attrs==21.4.0
-backports.functools-lru-cache==1.6.4
+backports.functools-lru-cache==1.6.6
 configparser==4.0.2
 contextlib2==0.6.0.post1
 coverage==5.5
@@ -23,8 +23,8 @@ more-itertools==5.0.0
 msgpack==1.0.5
 opentracing==2.4.0
 packaging==20.9
-parse-type==0.6.0
-parse==1.19.0
+parse-type==0.6.2
+parse==1.19.1
 pathlib2==2.3.7.post1
 pluggy==0.13.1
 py==1.11.0

.riot/requirements/e1fad7a.txt renamed to .riot/requirements/4e21258.txt

@@ -2,29 +2,30 @@
 # This file is autogenerated by pip-compile with Python 3.8
 # by the following command:
 #
-#    pip-compile --no-annotate --resolver=backtracking .riot/requirements/e1fad7a.in
+#    pip-compile --no-annotate --resolver=backtracking .riot/requirements/4e21258.in
 #
-attrs==22.2.0
-coverage[toml]==7.2.2
-exceptiongroup==1.1.1
+attrs==23.1.0
+coverage[toml]==7.2.7
+exceptiongroup==1.1.2
 glob2==0.7
 hypothesis==6.45.0
 iniconfig==2.0.0
 mako==1.2.4
-markupsafe==2.1.2
-mock==5.0.1
+markupsafe==2.1.3
+mock==5.0.2
 more-itertools==8.10.0
 msgpack==1.0.5
 opentracing==2.4.0
-packaging==23.0
-parse==1.19.0
-parse-type==0.6.0
-pluggy==1.0.0
+packaging==23.1
+parse==1.19.1
+parse-type==0.6.2
+pluggy==1.2.0
 py==1.11.0
-pytest==7.2.2
+pytest==7.4.0
 pytest-bdd==4.1.0
-pytest-cov==4.0.0
-pytest-mock==3.10.0
+pytest-cov==4.1.0
+pytest-mock==3.11.1
 six==1.16.0
 sortedcontainers==2.4.0
 tomli==2.0.1
+typing-extensions==4.7.1