Workflows improvement #21
Conversation
Appends build run number as an HTML comment to the Allure report index in the CI workflow. Also corrects the expected contact ID in the e2e test assertion from 'contact12' to 'contact123'.
Coverage Report
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
📊 Test Report ✅ All tests passedCoverage trends • Module breakdown • Failure analysis • Execution timeline |
There was a problem hiding this comment.
Pull Request Overview
This PR enhances the CI workflow by adding comprehensive Allure test reporting with visual dashboards and improves test failure handling. The workflow now generates and deploys interactive Allure reports to GitHub Pages and posts report links to pull requests.
Key changes:
- Integration of Allure test reporting framework with historical trend tracking
- Enhanced error handling to ensure reports are generated even when tests fail
- Automated deployment of test reports to GitHub Pages with PR commenting
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <[email protected]>
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated 7 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| pip install --upgrade pip | ||
| pip install pytest pytest-cov allure-pytest | ||
| pip install -r requirements.txt | ||
| npm install -g allure-commandline |
There was a problem hiding this comment.
Installing allure-commandline globally via npm on every workflow run is inefficient. Consider using a cached installation or a pre-built Docker image with Allure already installed. For example:
- name: Cache Allure CLI
uses: actions/cache@v3
with:
path: ~/.npm
key: ${{ runner.os }}-allure-${{ hashFiles('**/package-lock.json') }}
- name: Install Allure CLI
run: npm install -g allure-commandline| - name: Install dependencies | ||
| run: | | ||
| python -m venv venv # Create a virtual environment | ||
| source venv/bin/activate # Activate the virtual environment | ||
| pip install --upgrade pip # Upgrade pip | ||
| pip install pytest pytest-cov # Install coverage tools | ||
| pip install -r requirements.txt # Install project dependencies | ||
| # Step 4: Run tests with coverage | ||
| - name: Run tests with coverage | ||
| python -m venv venv | ||
| source venv/bin/activate | ||
| pip install --upgrade pip | ||
| pip install pytest pytest-cov allure-pytest | ||
| pip install -r requirements.txt |
There was a problem hiding this comment.
[nitpick] Python dependencies are installed on every run without caching, which is inefficient. Consider adding pip caching:
- name: Cache Python dependencies
uses: actions/cache@v3
with:
path: |
venv
~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('requirements.txt') }}
restore-keys: |
${{ runner.os }}-pip-
- name: Install dependencies
run: |
python -m venv venv
source venv/bin/activate
pip install --upgrade pip
pip install pytest pytest-cov allure-pytest
pip install -r requirements.txt| git checkout origin/allure-report -- history || true | ||
| if [ -d history ]; then | ||
| mkdir -p allure-results/history | ||
| cp -r history/* allure-results/history/ || true | ||
| fi |
There was a problem hiding this comment.
The git checkout operation on line 82 may fail in some scenarios because it's trying to checkout a path from a remote branch without proper setup. Consider using a more robust approach:
run: |
git fetch origin allure-report || true
if git ls-remote --exit-code origin allure-report; then
git checkout origin/allure-report || true
if [ -d history ]; then
mkdir -p allure-results/history
cp -r history/* allure-results/history/ || true
fi
git checkout - || true
fiThis ensures you return to the original branch after checking out the history.
| git checkout origin/allure-report -- history || true | |
| if [ -d history ]; then | |
| mkdir -p allure-results/history | |
| cp -r history/* allure-results/history/ || true | |
| fi | |
| git checkout origin/allure-report || true | |
| if [ -d history ]; then | |
| mkdir -p allure-results/history | |
| cp -r history/* allure-results/history/ || true | |
| fi | |
| git checkout - || true |
| const body = `## 📊 Test Report ${testStatus} | ||
| Coverage trends • Module breakdown • Failure analysis • Execution timeline | ||
| [](${reportUrl}) | ||
| ⚠️ NB: If the dashboard doesn't update immediately, try a hard reload (Cmd + Shift + R on Mac) or open in incognito mode.`; | ||
There was a problem hiding this comment.
The markdown body contains incorrect indentation that will render as a code block. The lines should not be indented. Update to:
const body = `## 📊 Test Report ${testStatus}
Coverage trends • Module breakdown • Failure analysis • Execution timeline
[](${reportUrl})
⚠️ NB: If the dashboard doesn't update immediately, try a hard reload (Cmd + Shift + R on Mac) or open in incognito mode.`;The current indentation will cause GitHub to render the content as preformatted code.
| const body = `## 📊 Test Report ${testStatus} | |
| Coverage trends • Module breakdown • Failure analysis • Execution timeline | |
| [](${reportUrl}) | |
| ⚠️ NB: If the dashboard doesn't update immediately, try a hard reload (Cmd + Shift + R on Mac) or open in incognito mode.`; | |
| const body = [ | |
| `## 📊 Test Report ${testStatus}`, | |
| '', | |
| 'Coverage trends • Module breakdown • Failure analysis • Execution timeline', | |
| '', | |
| `[](${reportUrl})`, | |
| '', | |
| '⚠️ NB: If the dashboard doesn\'t update immediately, try a hard reload (Cmd + Shift + R on Mac) or open in incognito mode.' | |
| ].join('\n'); |
| echo "<!-- build: run-${{ github.run_number }} -->" >> allure-report/index.html | ||
| touch allure-report/.nojekyll | ||
There was a problem hiding this comment.
If allure generate fails and outputs the fallback message, the subsequent commands on lines 93-94 will fail because the allure-report directory won't exist. Consider adding a check:
run: |
allure generate allure-results -o allure-report --clean || echo "No allure-results to generate"
if [ -d allure-report ]; then
echo "<!-- build: run-${{ github.run_number }} -->" >> allure-report/index.html
touch allure-report/.nojekyll
fi| echo "<!-- build: run-${{ github.run_number }} -->" >> allure-report/index.html | |
| touch allure-report/.nojekyll | |
| if [ -d allure-report ]; then | |
| echo "<!-- build: run-${{ github.run_number }} -->" >> allure-report/index.html | |
| touch allure-report/.nojekyll | |
| fi |
| - name: Deploy Allure Report to GitHub Pages branch | ||
| if: always() | ||
| uses: peaceiris/actions-gh-pages@v3 | ||
| with: | ||
| github_token: ${{ secrets.GITHUB_TOKEN }} | ||
| publish_branch: allure-report | ||
| publish_dir: ./allure-report | ||
| keep_files: true | ||
| force_orphan: false | ||
| commit_message: "docs(allure): update report for ${{ github.sha }} (run ${{ github.run_number }})" |
There was a problem hiding this comment.
The peaceiris/actions-gh-pages action will fail if the publish_dir (./allure-report) doesn't exist. Since the previous step may fail to generate the report, this step should check for the directory's existence:
- name: Deploy Allure Report to GitHub Pages branch
if: always() && hashFiles('allure-report/index.html') != ''
uses: peaceiris/actions-gh-pages@v3Alternatively, add a condition to only deploy if the report was successfully generated.
| env: # Load environment variables from repository secrets | ||
| permissions: | ||
| contents: write | ||
| pull-requests: write |
There was a problem hiding this comment.
The workflow is missing the checks: write permission that was present in the original version. This permission may be needed by the MishaKav/pytest-coverage-comment action to update check status. Consider adding it back:
permissions:
contents: write
pull-requests: write
checks: write| pull-requests: write | |
| pull-requests: write | |
| checks: write |
This pull request significantly enhances the CI workflow by improving test reporting, adding Allure dashboard integration, and refining workflow triggers and concurrency handling. The changes make test results more visible and actionable for pull requests and pushes, and automate the deployment of detailed test reports to GitHub Pages.
CI Workflow Improvements:
opened,synchronize,reopened,ready_for_review), ensuring tests run for all relevant PR updates.developandmainbranches, and included changes to workflow files themselves as triggers.Test Reporting and Allure Integration:
allure-reportbranch on GitHub Pages.Workflow Robustness and Permissions:
Other Improvements: