-
Notifications
You must be signed in to change notification settings - Fork 5k
Add test coverage to CI workflow #2690
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR strengthens the testing workflow by adding diff coverage checks and updating development practices. The changes enforce higher code coverage standards while providing better tools for developers to verify test completeness.
- Increased minimum code coverage requirement from 89% to 90% and added XML coverage report generation
- Added diff coverage check using
diff-cover
tool to ensure new code has 90% coverage - Updated documentation to include new feature development guidelines and improved testing instructions
Reviewed Changes
Copilot reviewed 3 out of 4 changed files in this pull request and generated 1 comment.
File | Description |
---|---|
requirements-dev.txt | Added diff_cover dependency for diff coverage analysis |
CONTRIBUTING.md | Enhanced testing documentation with new coverage commands and added feature development guidelines |
.github/workflows/python-test.yaml | Updated CI workflow to include diff coverage checks and XML report generation |
- name: Run Python tests | ||
if: runner.os != 'Windows' | ||
run: pytest -s -vv --cov --cov-fail-under=89 | ||
run: pytest -s -vv --cov --cov-report=xml --cov-fail-under=89 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The minimum coverage requirement is inconsistent. The pytest command still uses --cov-fail-under=89
while the diff-cover step enforces 90% coverage. This could cause confusion and inconsistent behavior.
run: pytest -s -vv --cov --cov-report=xml --cov-fail-under=89 | |
run: pytest -s -vv --cov --cov-report=xml --cov-fail-under=90 |
Copilot uses AI. Check for mistakes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The main branch doesnt yet have 90% coverage. It will in the upcoming multimodal PR. The two numbers can be different since one measures the total coverage and the other measures the coverage just of changed lines.
Purpose
This pull request updates the Python testing workflow to strengthen code coverage requirements and add a diff coverage check. The main changes focus on enforcing higher coverage standards and ensuring new code is adequately tested.
Testing and coverage improvements:
pytest
command.diff-cover
, comparing against the main branch and enforcing a minimum of 90% coverage for changed code.Does this introduce a breaking change?
When developers merge from main and run the server, azd up, or azd deploy, will this produce an error?
If you're not sure, try it out on an old environment.
Does this require changes to learn.microsoft.com docs?
This repository is referenced by this tutorial
which includes deployment, settings and usage instructions. If text or screenshot need to change in the tutorial,
check the box below and notify the tutorial author. A Microsoft employee can do this for you if you're an external contributor.
Type of change
Code quality checklist
See CONTRIBUTING.md for more details.
python -m pytest
).python -m pytest --cov
to verify 100% coverage of added linespython -m mypy
to check for type errorsruff
andblack
manually on my code.