feat: mark all integration tests as flaky with auto-retry

[twangodev]

Summary by CodeRabbit

• Tests
  • Introduced automatic retry functionality for integration tests to enhance stability and reduce flakiness
  • Cleaned up WebSocket integration tests by removing manual flaky test decorators

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

📝 Walkthrough

Walkthrough

A pytest hook is added to centrally mark integration tests for automatic retries with flakiness tolerance, while individual flaky decorators are removed from specific integration tests to eliminate redundancy.

Changes

Cohort / File(s)	Summary
Integration test configuration tests/integration/conftest.py	Added pytest_collection_modifyitems hook to automatically mark tests in paths containing "integration" with pytest.mark.flaky(reruns=9, reruns_delay=1) for automatic retry handling
Integration test decorators tests/integration/test_tts_websocket_integration.py	Removed individual @pytest.mark.flaky decorators from test_basic_websocket_streaming and test_async_websocket_streaming_with_model tests

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Poem

🐰 Tests now retry with centralized care,
No more decorators scattered everywhere!
Nine times we leap through flaky skies,
Until integration's steady prize. 🎯

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the main change: adding automatic retry capability to integration tests via pytest hooks and removing redundant decorators.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull request overview

This PR centralizes the flaky test marker configuration by implementing a pytest hook in the integration test conftest.py file, which automatically marks all integration tests as flaky with 9 retries and a 1-second delay between retries.

• Adds a pytest_collection_modifyitems hook to automatically mark all integration tests as flaky
• Removes individual @pytest.mark.flaky decorators from specific test methods in the WebSocket integration tests

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
tests/integration/conftest.py	Implements a pytest hook to automatically apply the flaky marker to all tests in the integration directory
tests/integration/test_tts_websocket_integration.py	Removes two individual flaky decorators from test methods, now covered by the centralized hook

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[coderabbitai]

Actionable comments posted: 0

🧹 Nitpick comments (1)

tests/integration/conftest.py (1)

16-21: Refine the path check and add type hints.

The centralized approach to marking integration tests as flaky is excellent. The pytest-rerunfailures dependency is already included in your dev dependencies. However, two refinements would improve the code:

1. Path matching robustness: The substring check "integration" in str(item.fspath) could unintentionally match other paths like tests named "integration-feature" elsewhere in the codebase. Use a more explicit check:

♻️ Suggested improvements

# Option 1: Check for the specific directory
if "tests/integration" in str(item.fspath).replace("\\", "/"):
    item.add_marker(pytest.mark.flaky(reruns=9, reruns_delay=1))

# Option 2: Use pathlib for cross-platform robustness (Path is already imported)
from pathlib import Path
if "integration" in Path(str(item.fspath)).parts:
    item.add_marker(pytest.mark.flaky(reruns=9, reruns_delay=1))

2. Type hints: Add type hints to the items parameter and return type for better code clarity:

def pytest_collection_modifyitems(items: list) -> None:

📜 Review details

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 6cb173c and 9c4795a.

📒 Files selected for processing (2)

• tests/integration/conftest.py
• tests/integration/test_tts_websocket_integration.py

💤 Files with no reviewable changes (1)

• tests/integration/test_tts_websocket_integration.py

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)

• GitHub Check: Integration Tests
• GitHub Check: Integration Tests