Pydantic benchmark plugin fix

[KRRT7]

PR Type

Enhancement, Bug fix

---

Description

• Add comprehensive benchmark CLI options

• Mark certain flags as ignored when tracing

• Remove direct benchmark_disable assignment

• Suppress codeflash plugin in subprocess discovery

---

Diagram Walkthrough

flowchart LR
  plugin_py["plugin.py: define benchmark_options"] --> addopt_loop["pytest_addoption loop adds options"]
  addopt_loop --> help_suffix["Append ignored suffix when tracing"]
  discovery_py["pytest_new_process_discovery.py"] --> suppress_flag["Add -p no:codeflash-benchmark"]

Loading

File Walkthrough

	Relevant files
Enhancement	plugin.py Add benchmark CLI options and tracing behavior                      codeflash-benchmark/codeflash_benchmark/plugin.py Introduce benchmark_options list of tuples Loop through options to add to parser Append "(ignored when --codeflash-trace)" suffix Remove direct config.option.benchmark_disable call +29/-1
Bug fix	pytest_new_process_discovery.py Suppress codeflash plugin in subprocess                                    codeflash/discovery/pytest_new_process_discovery.py Include -p no:codeflash-benchmark flag Prevent plugin interference in subprocess tests +1/-1

---

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Help text inconsistency The logic for appending the "(ignored when --codeflash-trace is used)" suffix excludes options containing "disable" or "skip", which contradicts the comment stating all benchmark options are ignored when tracing. This may mislead users. # These options are ignored when --codeflash-trace is used for option, action, default, help_text in benchmark_options: help_suffix = ( " (ignored when --codeflash-trace is used)" if "disable" not in option and "skip" not in option else "" ) parser.addoption(option, action=action, default=default, help=help_text + help_suffix) Options still applied Benchmark options are only annotated as ignored via help text but still parsed and applied when --codeflash-trace is used. Consider programmatically ignoring or blocking them at runtime. def pytest_addoption(parser: pytest.Parser) -> None: parser.addoption( "--codeflash-trace", action="store_true", default=False, help="Enable CodeFlash tracing for benchmarks" ) # These options are ignored when --codeflash-trace is used for option, action, default, help_text in benchmark_options: help_suffix = ( " (ignored when --codeflash-trace is used)" if "disable" not in option and "skip" not in option else "" ) parser.addoption(option, action=action, default=default, help=help_text + help_suffix) Plugin name verification The suppression flag -p no:codeflash-benchmark assumes the plugin is named "codeflash-benchmark". Verify this matches the actual plugin identifier to ensure it is correctly disabled during subprocess discovery. [tests_root, "-p no:logging", "--collect-only", "-m", "not skip", "-p", "no:codeflash-benchmark"], plugins=[PytestCollectionPlugin()]

[github-actions]

PR Code Suggestions ✨

No code suggestions found for the PR.