Skip to content

Commit dba2b3e

Browse files
authored
ci(appsec): add flaky decorator to failing langchain tests (#13435)
## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)
1 parent a966d1a commit dba2b3e

File tree

1 file changed

+7
-0
lines changed

1 file changed

+7
-0
lines changed

tests/appsec/integrations/langchain_tests/test_iast_langchain.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@
2929
from ddtrace.appsec._iast.constants import VULN_CMDI
3030
from tests.appsec.iast.conftest import iast_span_defaults # noqa: F401
3131
from tests.appsec.iast.taint_sinks.conftest import _get_span_report
32+
from tests.utils import flaky
3233

3334

3435
TEST_FILE = "tests/appsec/integrations/langchain_tests/test_iast_langchain.py"
@@ -78,27 +79,31 @@ def test_openai_functions_agent_output_parser(iast_span_defaults): # noqa: F811
7879
assert is_pyobject_tainted(result.tool_input["arg1"])
7980

8081

82+
@flaky(until=1752674136)
8183
def test_llm_generate(iast_span_defaults): # noqa: F811
8284
llm = FakeListLLM(responses=["I am a fake LLM"])
8385
prompt = prepare_tainted_prompt()
8486
result = llm.invoke(prompt)
8587
assert is_pyobject_tainted(result)
8688

8789

90+
@flaky(until=1752674136)
8891
async def test_llm_agenerate(iast_span_defaults): # noqa: F811
8992
llm = FakeListLLM(responses=["I am a fake LLM"])
9093
prompt = prepare_tainted_prompt()
9194
result = await llm.ainvoke(prompt)
9295
assert is_pyobject_tainted(result)
9396

9497

98+
@flaky(until=1752674136)
9599
def test_chatmodel_generate(iast_span_defaults): # noqa: F811
96100
chatmodel = FakeListChatModel(responses=["I am a fake chat model"])
97101
prompt = prepare_tainted_prompt()
98102
result = chatmodel.invoke(prompt)
99103
assert is_pyobject_tainted(result.content)
100104

101105

106+
@flaky(until=1752674136)
102107
async def test_chatmodel_agenerate(iast_span_defaults): # noqa: F811
103108
chatmodel = FakeListChatModel(responses=["I am a fake chat model"])
104109
prompt = prepare_tainted_prompt()
@@ -138,6 +143,7 @@ async def test_cmdi_with_shelltool_ainvoke(iast_span_defaults): # noqa: F811
138143
assert span_report
139144

140145

146+
@flaky(until=1752674136)
141147
def test_cmdi_with_agent_invoke(iast_span_defaults): # noqa: F811
142148
agent = prepare_cmdi_agent()
143149
prompt = prepare_tainted_prompt()
@@ -152,6 +158,7 @@ def test_cmdi_with_agent_invoke(iast_span_defaults): # noqa: F811
152158
assert "class" not in location
153159

154160

161+
@flaky(until=1752674136)
155162
async def test_cmdi_with_agent_ainvoke(iast_span_defaults): # noqa: F811
156163
agent = prepare_cmdi_agent()
157164
prompt = prepare_tainted_prompt()

0 commit comments

Comments
 (0)