Skip to content

Conversation

@karangattu
Copy link
Collaborator

@karangattu karangattu commented Sep 9, 2025

This pull request updates the testing dependencies and configuration for the project, primarily by renaming and expanding the test dependency group and correcting a model name in the test generation code. The most important changes are:

  • Renamed the testgen dependency group to add-test in pyproject.toml and expanded it to include pytest, pytest-playwright, and playwright for improved test capabilities.
  • Updated the Makefile to use the new add-test dependency group instead of the old testgen group for installing test dependencies.
  • Fixed the default OpenAI model name in shiny/pytest/_generate/_main.py from gpt-5-mini-2025-08-07 to gpt-5-2025-08-07 to ensure correct model usage.

Replaces the 'testgen' extra with 'add-test' in pyproject.toml and updates the Makefile to use the new extra for installing AI-related test dependencies. Additional test dependencies were added to the 'add-test' extra.
Changed DEFAULT_OPENAI_MODEL from 'gpt-5-mini-2025-08-07' to 'gpt-5-2025-08-07' in the Config class to reflect the correct model identifier.
@github-actions
Copy link

github-actions bot commented Sep 9, 2025

Test Generation Evaluation Results (Averaged across 3 attempts)

🔍 Inspect AI Test Quality Evaluation

  • Complete (C): 8.3
  • Partial (P): 0.3
  • Incomplete (I): 0.3
  • Passing Rate: 8.6/9.0 (96.3%)
  • Quality Gate: ✅ PASSED (≥80% required)

🎯 Overall Result

✅ PASSED - Quality gate based on Inspect AI results


Results are averaged across 3 evaluation attempts for improved reliability.

@karangattu karangattu requested a review from schloerke September 9, 2025 17:39
@karangattu karangattu merged commit 031d1ae into main Sep 9, 2025
68 checks passed
@karangattu karangattu deleted the rename-testgen branch September 9, 2025 19:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants