Skip to content

Commit 14a9dc8

Browse files
committed
Switch PR comment to sticky-pull-request-comment action
Replaces the use of actions/github-script with marocchino/sticky-pull-request-comment for posting AI evaluation results on pull requests. Adds a step to prepare the comment body and writes it to a file, improving error handling and ensuring comments are updated rather than duplicated.
1 parent a6919e2 commit 14a9dc8

File tree

1 file changed

+45
-37
lines changed

1 file changed

+45
-37
lines changed

.github/workflows/validate-test-generation-prompts.yaml

Lines changed: 45 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -106,43 +106,51 @@ jobs:
106106
exit 1
107107
fi
108108
109+
- name: Prepare Comment Body
110+
if: github.event_name == 'pull_request'
111+
run: |
112+
if [ -f "results/summary.json" ]; then
113+
python -c "
114+
import json
115+
import os
116+
117+
try:
118+
with open('results/summary.json', 'r') as f:
119+
results = json.load(f)
120+
121+
comment = f'''## Inspect AI Evaluation Results
122+
123+
- **Tests Passed**: {results['passed']}/{results['total']}
124+
- **Quality Gate**: {'✅ PASSED' if results['quality_gate_passed'] else '❌ FAILED'}
125+
126+
### Details
127+
{results['details']}
128+
'''
129+
130+
with open('comment_body.txt', 'w') as f:
131+
f.write(comment)
132+
except Exception as e:
133+
print(f'Error reading summary file: {e}')
134+
comment = '''## Inspect AI Evaluation Results
135+
136+
❌ **Error**: Could not read evaluation results summary file.
137+
138+
Please check the workflow logs for details.'''
139+
140+
with open('comment_body.txt', 'w') as f:
141+
f.write(comment)
142+
"
143+
else
144+
echo "## Inspect AI Evaluation Results
145+
146+
❌ **Error**: Could not read evaluation results summary file.
147+
148+
Please check the workflow logs for details." > comment_body.txt
149+
fi
150+
109151
- name: Comment PR Results
110152
if: github.event_name == 'pull_request'
111-
uses: actions/github-script@v7
153+
uses: marocchino/sticky-pull-request-comment@v2
112154
with:
113-
script: |
114-
const fs = require('fs');
115-
116-
try {
117-
const results = JSON.parse(fs.readFileSync('results/summary.json', 'utf8'));
118-
119-
const comment = \`## Inspect AI Evaluation Results
120-
121-
- **Tests Passed**: \${results.passed}/\${results.total}
122-
- **Quality Gate**: \${results.quality_gate_passed ? '✅ PASSED' : '❌ FAILED'}
123-
124-
### Details
125-
\${results.details}
126-
\`;
127-
128-
github.rest.issues.createComment({
129-
issue_number: context.issue.number,
130-
owner: context.repo.owner,
131-
repo: context.repo.repo,
132-
body: comment
133-
});
134-
} catch (error) {
135-
console.error('Error reading summary file:', error);
136-
const comment = \`## Inspect AI Evaluation Results
137-
138-
❌ **Error**: Could not read evaluation results summary file.
139-
140-
Please check the workflow logs for details.\`;
141-
142-
github.rest.issues.createComment({
143-
issue_number: context.issue.number,
144-
owner: context.repo.owner,
145-
repo: context.repo.repo,
146-
body: comment
147-
});
148-
}
155+
header: inspect-ai-results
156+
path: comment_body.txt

0 commit comments

Comments
 (0)