Skip to content

Conversation

ChrisRackauckas
Copy link
Member

Summary

Improves error messages when LinearSolveAutotune fails to upload benchmark results to GitHub, making it easier to diagnose and fix authentication/permission issues.

Motivation

As seen in issue #669, when the autotune system fails to upload results, the error messages don't provide enough information to understand why it failed. This makes it difficult for users to fix the issue.

Changes

Enhanced Error Reporting

  • Changed from @warn to @error for critical failures
  • Added detailed error information including:
    • Repository and issue number
    • Authentication method used
    • Error type and full error message

Specific Error Guidance

Added pattern matching for common HTTP errors with actionable advice:

  • 403 Forbidden: Suggests checking repository permissions, token scopes, or commenting restrictions
  • 404 Not Found: Indicates the issue may have been deleted or moved
  • 401 Unauthorized: Suggests re-authenticating with gh auth login
  • Rate Limits: Advises waiting and retrying later

Improved Error Propagation

  • Modified comment_on_issue_api and comment_on_issue_gh to properly propagate errors
  • Added debug logging for detailed diagnostics
  • Better error context in gh CLI failures

User-Friendly Fallback

When upload fails and results are saved locally:

  • Now includes the direct GitHub issue URL for manual sharing
  • Clear instructions on how to manually upload the results

Example Output

Before:

[ Info: ❌ Failed to add comment to GitHub issue: <error>
[ Info: 💡 This could be due to network issues, repository permissions, or API limits.
[ Info: 📁 Results saved locally to autotune_results_2025-08-09T19-26-43.232.md

After:

┌ Error: ❌ Failed to add comment to GitHub issue #669
│    Repository: SciML/LinearSolve.jl
│    Auth method: gh_cli
│    Error type: ProcessFailedException
│    Error message: gh CLI error: HTTP 403: Resource not accessible by integration
└ @ LinearSolveAutotune
[ Info: 📝 This appears to be a permissions issue. Possible causes:
[ Info:     1. You may not have write access to SciML/LinearSolve.jl
[ Info:     2. Your token may lack the 'public_repo' or 'repo' scope
[ Info:     3. The repository may have restricted commenting
[ Info:     Try: gh auth status to check your authentication
[ Info: 📁 Results saved locally to autotune_results_2025-08-09T19-26-43.232.md as backup
[ Info:     You can manually share this file on the issue tracker:
[ Info:     https://github.com/SciML/LinearSolve.jl/issues/669

Test Plan

  • Test with invalid authentication
  • Test with valid auth but no permissions
  • Verify error messages are clear and actionable
  • CI tests should pass

This improvement will help users understand and resolve upload failures more effectively.

🤖 Generated with Claude Code

Co-Authored-By: Claude [email protected]

- Add detailed error reporting in upload_to_github with specific error types
- Include repository, issue number, and auth method in error output
- Provide actionable guidance for common errors (403, 404, 401, rate limits)
- Show specific help messages for permission issues
- Include the GitHub issue URL in fallback message for manual sharing
- Improve error propagation in comment_on_issue functions
- Use @error for critical failures and @debug for detailed diagnostics

This helps users understand why uploads fail and what actions to take.
@ChrisRackauckas ChrisRackauckas merged commit b28caa3 into SciML:main Aug 9, 2025
100 of 118 checks passed
@ChrisRackauckas ChrisRackauckas deleted the improve-autotune-error-messages branch August 9, 2025 23:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant