Skip to content

Commit da1f899

Browse files
ci(llmobs): skip flaky test again [backport 3.14] (#14611)
Backport 8beb204 from #14607 to 3.14. skips flaky test where a cassette (fixture) was removed. we'll be implementing a better CI solution in the testagent to later re-enable this test (plus the others that had cassettes generated but weren't unskipped (oops 😅)) ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) Co-authored-by: Sam Brenner <[email protected]>
1 parent bea8ea8 commit da1f899

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

tests/llmobs/test_llmobs_evaluator_runner.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -92,6 +92,7 @@ def test_evaluator_runner_multiple_evaluators(llmobs, mock_llmobs_eval_metric_wr
9292
]
9393

9494

95+
@pytest.mark.skip(reason="Skipping due to flakiness in hitting the staging endpoint")
9596
def test_evaluator_runner_on_exit(mock_writer_logs, run_python_code_in_subprocess):
9697
env = os.environ.copy()
9798
pypath = [os.path.dirname(os.path.dirname(os.path.dirname(__file__)))]

0 commit comments

Comments
 (0)